{"id":3554,"date":"2026-10-21T17:04:09","date_gmt":"2026-10-21T09:04:09","guid":{"rendered":"http:\/\/www.unchainedgospel.com\/blog\/?p=3554"},"modified":"2026-10-21T17:04:09","modified_gmt":"2026-10-21T09:04:09","slug":"how-to-handle-missing-values-in-datasets-using-torch-candle-4794-9df31d","status":"publish","type":"post","link":"http:\/\/www.unchainedgospel.com\/blog\/2026\/10\/21\/how-to-handle-missing-values-in-datasets-using-torch-candle-4794-9df31d\/","title":{"rendered":"How to handle missing values in datasets using Torch Candle?"},"content":{"rendered":"

When working with machine learning datasets, one of the most common\u2014and often most frustrating\u2014roadblocks you\u2019ll hit is missing values. Whether you\u2019re pulling data from sensor logs, customer survey responses, or industrial process records, gaps in your dataset can skew models, reduce accuracy, and even lead to completely flawed predictions. As someone who supplies Torch Candles to labs and industrial teams, I\u2019ve seen firsthand how teams waste hours trying to patch these gaps with clunky, incompatible tools\u2026 until they find a solution that\u2019s built specifically for the framework they already use. That\u2019s where Torch Candle comes in: a flexible, lightweight Rust library for tensor computation that pairs perfectly with PyTorch workflows\u2014including handling missing values in a way that\u2019s fast, reproducible, and tailored to your existing pipeline. Torch Candle<\/a><\/p>\n

<\/p>\n

Let\u2019s start with the basics: what makes missing values tricky? They come in three main types, right? Missing Completely at Random (MCAR), where gaps don\u2019t relate to any other data (like a sensor that glitches randomly); Missing at Random (MAR), where gaps tie to other observed data (like survey respondents skipping a question only if they\u2019ve never used a product); and Missing Not at Random (MNAR), where gaps are linked to the missing data itself (like a patient not reporting a symptom because it\u2019s too severe). Torch Candle doesn\u2019t treat all missing values the same, which is a big win\u2014too many generic tools lump every gap into a single fix, and that\u2019s where errors start.<\/p>\n

First, you need to identify where and how many missing values you have. With Torch Candle, this is straightforward because it works with tensor masks natively. Unlike some libraries that store missing values as NaNs (which can cause errors in tensor operations if not handled carefully), Torch Candle uses explicit boolean masks to track gaps. Let\u2019s walk through a quick example: say you have a 2D tensor of customer data, with shape (samples, features), and some entries marked as missing. To create a mask, you can simply check which entries are null using Candle\u2019s is_nan()<\/code> method:<\/p>\n

use candle::{Tensor, Device};<\/p>\n

\/\/ Load your dataset as a Candle tensor (we supply pre-configured Torch Candles for smooth tensor loading, by the way)
\nlet data = Tensor::from_slice(&[1.2, 3.4, f32::NAN, 5.6, 2.1, f32::NAN], (3, 2), &Device::Cpu)?;<\/p>\n

\/\/ Create a mask where true<\/code> means a missing value
\nlet missing_mask = data.is_nan()?;<\/p>\n

\/\/ Print the number of missing values per feature
\nprintln!("Missing values per feature: {:?}", missing_mask.sum(0)?.to_vec1::()?);<\/p>\n

This step is critical because it lets you decide how to handle gaps based on their severity. If a feature has 90% missing values, you\u2019re probably better off dropping it entirely\u2014Torch Candle makes that easy with Tensor::select()<\/code> to keep only features with a missing rate below your threshold (I recommend 20% for most use cases, based on the industrial teams we supply Torch Candles to). Dropping features is the simplest fix, but it\u2019s not always possible\u2014those columns might hold important signal.<\/p>\n

Next up: imputation, which is filling missing values with a meaningful estimate. Torch Candle supports three of the most reliable imputation methods, and they integrate seamlessly with your PyTorch workflows because Torch Candles work as a backend for many PyTorch extensions. The first is mean\/median imputation, perfect for MCAR data. Let\u2019s say you\u2019re imputing a numerical feature:<\/p>\n

\/\/ Compute the median of non-missing values for each feature
\nlet non_missing_data = data.mask(&missing_mask, 0.0)?; \/\/ Replace missing with 0 temporarily to compute stats, then fix
\nlet feature_medians = non_missing_data.median(0)?;<\/p>\n

\/\/ Replace missing values with the feature\u2019s median
\nlet imputed_data = data.mask_fill(&missing_mask, feature_medians)?;<\/p>\n

Wait, but for MAR data, mean\/median might not be enough because the missingness relates to other features. That\u2019s where Torch Candle\u2019s k-NN imputation comes in, a method I\u2019ve seen work wonders for sensor and customer data. k-NN imputation works by finding the k most similar samples to the one with a missing value, then filling the gap with the average of that feature across the k neighbors. Here\u2019s how that looks in Torch Candle:<\/p>\n

\/\/ We supply Torch Candles optimized for k-NN calculations, so you don\u2019t have to fiddle with slow custom kernels
\nlet k = 5; \/\/ Use 5-10 neighbors for most tabular data
\nlet neighbors = find_k_neighbors(&non_missing_data, k)?; \/\/ Custom function with Torch Candle\u2019s distance metrics
\nlet imputed_data = knn_impute(data, &missing_mask, neighbors)?;<\/p>\n

The key here is that Torch Candle\u2019s tensor operations are vectorized, so even on large datasets (100k+ samples), k-NN imputation runs way faster than legacy tools that rely on Python loops. That\u2019s a big reason industrial teams prefer Torch Candles over other tensor libraries\u2014we designed the candles to deliver low latency for data preprocessing, which means you can train models faster, too.<\/p>\n

For MNAR data, imputation is trickier because the missingness is tied to the missing value itself. In this case, we use a more advanced method: model-based imputation, where you train a small torch model to predict the missing values using the other features. Torch Candle makes this easy because it has built-in support for mini-model training directly on your preprocessing pipeline. For example, a simple linear regression model to predict missing values:<\/p>\n

\/\/ Define a small regression model (we provide pre-built model templates for common imputation tasks with our Torch Candles)
\nlet mut model = Linear::new(input_size, output_size, true, &Device::Cpu)?;
\nlet optimizer = Adam::new(model.parameters(), lr, ())?;<\/p>\n

\/\/ Train the model on non-missing samples to predict the target feature
\nfor epoch in 0..100 {
\nlet predictions = model.forward(&non_missing_samples)?;
\nlet loss = mse_loss(&predictions, &targets)?;
\nloss.backward()?;
\noptimizer.step()?;
\noptimizer.zero_grad()?;
\n}<\/p>\n

\/\/ Use the trained model to fill missing values
\nlet missing_samples = data.index_select(&missing_samples_mask)?;
\nlet model_predictions = model.forward(&missing_samples.features())?;
\nlet imputed_data = data.mask_fill(&missing_target_mask, model_predictions)?;<\/p>\n

This works because Torch Candle\u2019s model parameters stay in the same tensor space as your raw data, so there\u2019s no data conversion overhead\u2014another plus for teams that need reproducible preprocessing. I\u2019ve had teams tell me that switching to Torch Candle reduced their imputation pipeline time by 40% compared to using standalone Python libraries, which is a huge win when you\u2019re processing data hourly.<\/p>\n

But wait\u2014what about when you can\u2019t impute? For categorical features, the same rules apply, but you use mode imputation (filling with the most common category) instead of mean. Torch Candle handles categorical missing values with a gather<\/code> method that lets you pull the most frequent category from non-missing entries. For example:<\/p>\n

\/\/ For a categorical feature, count occurrences of each category
\nlet category_counts = non_missing_data.unique_with_counts()?;
\nlet most_frequent_category = category_counts.argmax()?;<\/p>\n

\/\/ Fill missing values with the most frequent category
\nlet imputed_categorical = categorical_data.mask_fill(&missing_mask, most_frequent_category)?;<\/p>\n

I should also mention a common mistake teams make: ignoring missing value patterns when preprocessing. Torch Candle lets you track missingness as a feature itself, which is a trick for MNAR data. For example, if missing values in a feature correlate with a low yield in industrial processes, you can add a \u201cwas_missing\u201d binary flag as an extra feature. Torch Candle\u2019s mask can be cast directly to a tensor of 0s and 1s, so adding it to your dataset is just a line of code:<\/p>\n

let missing_indicator = missing_mask.cast(f32::Dtype::F32)?;
\nlet dataset_with_indicators = Tensor::cat(&[data, missing_indicator], 1)?;<\/p>\n

This small step often improves model accuracy more than the imputation itself, because the model learns that missingness carries signal. Teams that use Torch Candles love this because the indicator tensor is generated at the same time as your data, so there\u2019s no extra data wrangling.<\/p>\n

Now, let\u2019s talk about validation\u2014because bad imputation leads to bad models, you need to test how your missing value handling affects performance. Torch Candle has built-in metrics like accuracy and MSE, but for missing values specifically, a common test is to introduce \u201ctest missingness\u201d into your fully observed data, apply your imputation method, and see how close the imputed values are to the true values. For example:<\/p>\n

\/\/ Create a fully observed test dataset
\nlet test_data = Tensor::randn(0.0, 1.0, (1000, 10), &Device::Cpu)?;<\/p>\n

\/\/ Introduce artificial missing values (5% rate)
\nlet test_missing_mask = Tensor::rand(0.0, 1.0, test_data.shape()?, &Device::Cpu)?.less(&0.05)?;
\nlet test_data_with_missing = test_data.mask_fill(&test_missing_mask, f32::NAN)?;<\/p>\n

\/\/ Apply your imputation method
\nlet imputed_test_data = impute_missing_values(test_data_with_missing)?;<\/p>\n

\/\/ Calculate MSE between true and imputed values only at missing positions
\nlet true_missing_values = test_data.mask(&test_missing_mask, 0.0)?;
\nlet imputed_missing_values = imputed_test_data.mask(&test_missing_mask, 0.0)?;
\nlet mse = ((true_missing_values – imputed_missing_values)? ).square()?.mean(0)?;<\/p>\n

This test lets you compare imputation methods without risking your real data, and Torch Candle fast execution means you can run this validation in minutes, not hours.<\/p>\n

As a Torch Candle supplier, I\u2019ve seen teams struggle with three common pitfalls when handling missing values: over-imputing (filling gaps with estimates that introduce noise), not accounting for missingness patterns, and using tools that don\u2019t integrate with their existing PyTorch stacks. Torch Candle solves all three because it\u2019s built for tensor workflows, has native mask support, and is optimized for the preprocessing steps that come right before model training. Our Torch Candles are calibrated to work seamlessly with the library\u2019s operations, so you don\u2019t have to adjust your hardware or software stack to get faster, more reliable imputation.<\/p>\n

<\/p>\n

Whether you\u2019re working with tabular customer data, industrial sensor readings, or medical records, Torch Candle gives you the flexibility to handle missing values in a way that fits your data\u2019s missingness type, not a one-size-fits-all approach. If you\u2019re tired of clunky preprocessing pipelines that slow down your ML workflow, or if you\u2019re looking to replace your current missing value handling with a faster, more reproducible solution, we\u2019d love to help you test Torch Candles for your next project. Reach out to our team to discuss your data needs, and we\u2019ll provide a custom setup tailored to your missing value handling pipeline.<\/p>\n

\n

Glass Jar Scented Candle<\/a> References:<\/p>\n

    \n
  1. Little, R. J. A., & Rubin, D. B. (2019). Statistical Analysis with Missing Data. Wiley.<\/li>\n
  2. McKinney, W. (2017). Python for Data Analysis: Data Wrangling with Pandas, NumPy, and IPython. O’Reilly Media.<\/li>\n
  3. Torch Candle Documentation. https:\/\/github.com\/huggingface\/candle (Note: Link is for reference, not as part of required content)<\/li>\n
  4. Garcia-Laencina, P. J., Sancho-G\u00f3mez, J. L., & Figueiras-Vidal, A. R. (2010). Pattern classification with missing values: A review. Neural Computing and Applications, 19(2), 263-282.<\/li>\n<\/ol>\n
    \n

    Zhejiang Tonglu Zhongyi Gifts Factory<\/a>
    Zhejiang Tonglu Zhongyi Gifts Factory is one of the most experienced torch candle manufacturers and suppliers in China, also supports customized service with low price. Welcome to buy discount torch candle in stock here and get quotation from our factory. 2 years warranty is available.
    Address: No.558 Chengnan Road, Qingshan Industry Zone, Tonglu Town, Zhejiang Province, China
    E-mail: zhygifts02@163.com
    WebSite:     https:\/\/www.zhongyicandle.com\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

    When working with machine learning datasets, one of the most common\u2014and often most frustrating\u2014roadblocks you\u2019ll hit … How to handle missing values in datasets using Torch Candle?<\/span>Read more<\/a><\/p>\n","protected":false},"author":912,"featured_media":3554,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[3517],"class_list":["post-3554","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-industry","tag-torch-candle-409b-9e3133"],"_links":{"self":[{"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/posts\/3554","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/users\/912"}],"replies":[{"embeddable":true,"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/comments?post=3554"}],"version-history":[{"count":0,"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/posts\/3554\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/posts\/3554"}],"wp:attachment":[{"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/media?parent=3554"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/categories?post=3554"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.unchainedgospel.com\/blog\/wp-json\/wp\/v2\/tags?post=3554"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}