Why Sample Size Matters in Production Case Studies
A Lean Six Sigma project can fail for a simple reason: the team makes a decision from too little data. A small sample may suggest that a process improved, a supplier changed, or a defect rate dropped, when the apparent change is only random process noise.
Sample size is the link between production data and trustworthy conclusions. It affects confidence intervals, hypothesis tests, control plans, capability studies, and the credibility of improvement claims. Too few observations create risky decisions. Too many can waste time, material, test capacity, and money.
The goal is not to collect the largest possible sample. The goal is to collect enough data to answer the project question with the required precision and risk level.
Why sample size matters in quality improvement
Lean Six Sigma teams use samples because measuring every unit is often impractical. A filling line may produce thousands of cans per hour. A bearing life test may take weeks or months. Destructive tests consume the product. In those situations, a sample provides evidence about the larger process.
Sample size affects several common project questions:
Is the process average close enough to target?
Did the improvement reduce variation?
Is the defect rate truly lower than before?
Can the process meet specification limits?
How much risk remains if the team changes the control plan?
A sample is useful because it estimates a population value, such as a mean, standard deviation, proportion defective, or percentile. But every estimate has sampling error. Larger samples reduce that error.
For example, if a soup line has a target net weight of 15.00 oz, a sample of 5 cans may show an average of 15.06 oz. That sounds high. A sample of 60 cans may show an average of 15.01 oz, which tells a different story. The larger sample gives a clearer view of the actual process center.
The same principle applies to bearing life. Three bearings might fail early and create alarm, while 40 tested bearings may show that early failures are rare but possible. The right sample size helps separate special cause signals from normal variation.
How the Central Limit Theorem supports production sampling
The Central Limit Theorem, often called the CLT, is one of the main reasons sampling works in quality improvement.
In practical terms, the CLT says that the distribution of sample means tends to become approximately normal as the sample size grows, even when the individual observations are not perfectly normal. This matters because many common confidence intervals and hypothesis tests depend on the behavior of sample averages.
For production data, the CLT helps answer questions such as:
How close is the sample average to the true process average?
What confidence interval should surround the measured mean?
How many observations are needed to estimate the mean within a chosen margin of error?
The CLT does not make bad data good. It does not fix biased sampling, mixed processes, poor measurement systems, or unstable conditions. It also does not mean every sample of 30 is automatically enough. The required sample size depends on the process variation, the required precision, the shape of the data, and the decision risk.
A useful way to think about the CLT is this: individual values may be messy, but averages become more predictable as sample size increases. That predictability is what allows a Lean Six Sigma team to make statistical statements about the process.
A step-by-step method for calculating sample size
Sample size calculation starts with the project question. The formula changes depending on whether the team is estimating a mean, comparing two means, estimating a defect rate, or testing reliability.
1. Define the decision the data must support
State the question in measurable terms.
Examples:
Estimate the average soup can fill weight within ±0.05 oz.
Detect a 0.10 oz change in average fill weight after a machine adjustment.
Estimate bearing life on a log scale within a chosen precision.
Compare bearing life between two heat treatment methods.
A vague question leads to a vague sample plan. A clear question defines the statistic and the sample size method.
2. Choose the response type
Most production data falls into one of these groups:
Data type | Common statistic | Example |
Continuous data | Mean or standard deviation | Fill weight, shaft diameter, torque |
Attribute data | Proportion defective | Pass or fail inspection |
Time-to-failure data | Median life, B10 life, reliability at time | Bearing life, motor life |
Continuous data often needs fewer samples than attribute data for the same decision quality. Reliability data often needs special planning because tests may be long, censored, or highly skewed.
3. Set confidence, power, and practical difference
Common planning inputs include:
Confidence level Often 95 percent for estimating a mean or proportion.
Margin of error The maximum acceptable half-width of the confidence interval.
Power The chance of detecting a real difference when one exists. Common targets are 80 percent or 90 percent.
Practical difference The smallest change that matters to the process or customer.
A statistically significant difference may not be operationally meaningful. A practical difference anchors the calculation in business and quality impact.
4. Estimate process variation
For a mean, the most common formula needs an estimate of the standard deviation. Sources may include:
Historical process data
A pilot sample
Similar product families
Prior validation studies
Engineering judgment, used only when data is limited
For a continuous mean, a common sample size formula is:
`n = (Z × σ / E)^2`
Where:
`n` is the required sample size
`Z` is the confidence value, such as 1.96 for 95 percent confidence
`σ` is the estimated process standard deviation
`E` is the desired margin of error
Round up to the next whole number. If the calculation gives 49.2, use 50.
5. Adjust for the sampling situation
The formula gives a starting point. The real sampling plan must also consider how production runs.
Check these points before collecting data:
Sample across normal sources of variation, such as shifts, cavities, lines, lots, operators, and raw material batches.
Avoid taking all samples from one short, unusually stable period.
Confirm the measurement system can detect the differences of interest.
Account for lost samples, censored tests, rework, or invalid measurements.
Use subgrouping when the project question concerns within-subgroup and between-subgroup variation.
The best statistical formula cannot rescue a biased sample.
How non-normal data changes sample size decisions
Many Lean Six Sigma tools assume normal data, or at least use methods that work well when sample averages are approximately normal. Production data does not always cooperate.
Common non-normal patterns include:
Right-skewed data, such as cycle time and bearing life
Left-skewed data near a physical upper limit
Bimodal data from mixed machines or suppliers
Count data with many zeros
Censored data from tests stopped before all units fail
Truncated data caused by inspection screening
Non-normal data affects sample size in several ways.
First, skewed data often needs a larger sample for the sample mean to behave predictably. The CLT still helps, but it may require more observations when the underlying data has a long tail.
Second, the mean may not be the best statistic. For bearing life, the median life or B10 life may be more useful than the arithmetic mean. In reliability work, B10 means the time by which 10 percent of units are expected to fail. That type of question needs reliability models rather than a simple mean formula.
Third, transformations may help. Lognormal data, for example, often becomes easier to analyze after taking natural logs. Bearing life frequently behaves better on a log scale than on the original hour scale.
Fourth, nonparametric methods may require more data because they make fewer assumptions. They are useful, but the cost is often a larger sample.
A practical rule is simple: if the histogram is heavily skewed, mixed, or censored, do not rely on a basic normal formula without review. Use a pilot sample, inspect the distribution, and choose a method that matches the data.
Example of soup can weight-to-weight variation
Assume a food manufacturer runs a soup filling process with a labeled net weight of 15.00 oz. The project team wants to estimate the average fill weight and understand can-to-can variation after a filler adjustment.
The quality risk has two sides. Underfilled cans can lead to compliance and customer issues. Overfilled cans give away product and increase cost. The team needs enough data to judge the process center with confidence.
Define the question
The team asks:
Can we estimate the true average fill weight within ±0.05 oz at 95 percent confidence?
From historical data, the standard deviation of individual can weights is about 0.18 oz.
Use the formula for estimating a mean
The formula is:
`n = (Z × σ / E)^2`
Use:
`Z = 1.96` for 95 percent confidence
`σ = 0.18 oz`
`E = 0.05 oz`
Calculation:
`n = (1.96 × 0.18 / 0.05)^2`
`n = (0.3528 / 0.05)^2`
`n = 7.056^2`
`n = 49.79`
Round up to 50 cans.
That means the team should sample at least 50 cans to estimate the process mean within ±0.05 oz, assuming the historical standard deviation is reasonable and the sample reflects normal production.
Improve the sampling plan
Taking 50 consecutive cans may be easy, but it may not represent the process. A better plan might collect:
10 cans from each of 5 time periods during the run
Samples across startup, steady-state production, and later production
Cans from all filler heads if the machine has multiple heads
This plan gives the team information about both the average weight and weight-to-weight variation across the run.
Detect a meaningful change
Suppose the team also wants to detect a 0.10 oz shift in average fill weight between the old and new settings with 90 percent power at a 95 percent confidence level.
A common two-sample planning formula is:
`n per group = 2 × (Zα/2 + Zβ)^2 × σ^2 / δ^2`
Use:
`Zα/2 = 1.96`
`Zβ = 1.28` for 90 percent power
`σ = 0.18 oz`
`δ = 0.10 oz`
Calculation:
`n per group = 2 × (1.96 + 1.28)^2 × 0.18^2 / 0.10^2`
`n per group = 2 × 3.24^2 × 0.0324 / 0.01`
`n per group ≈ 68`
The team would need about 68 cans before and 68 cans after the adjustment to detect a 0.10 oz difference with the chosen risk levels.
This example shows why sample size changes with the question. Estimating the current mean required 50 cans. Comparing before and after settings with strong power required about 68 per group.
Example of bearing life sample size
Bearing life is a different type of production data. It is usually right-skewed because some bearings fail early, many last near the center of the distribution, and a few last a very long time. A simple normal assumption on raw life hours may give poor results.
Assume an engineering team tests a bearing used in an industrial motor. The team wants to estimate typical life after a material change. A pilot study suggests the natural log of bearing life is roughly normal, with a standard deviation of 0.45 log units.
Define the question
The team asks:
How many bearings are needed to estimate the mean of log life within ±0.10 log units at 95 percent confidence?
This is not the same as saying ±0.10 hours. On the original life scale, log units translate into a percentage band around the geometric mean.
Calculate sample size on the log scale
Use the same mean estimation formula, but apply it to log life:
`n = (Z × σlog / E)^2`
Use:
`Z = 1.96`
`σlog = 0.45`
`E = 0.10`
Calculation:
`n = (1.96 × 0.45 / 0.10)^2`
`n = (0.882 / 0.10)^2`
`n = 8.82^2`
`n = 77.79`
Round up to 78 bearings.
That sample size estimates the average log life with the desired precision, assuming the lognormal model fits and the test observes enough failures.
Account for censoring and long test times
Bearing life studies often stop before every unit fails. A bearing that has not failed by the test end is censored. It still provides information, but not as much as a complete failure time.
If the team expects censoring, it may need to start more than 78 bearings to end with enough useful failure information. For example, if some bearings will still be running at the end of the test, the sample plan should include extra units or use a reliability test design based on expected censoring.
If the project focuses on B10 life rather than average log life, the team should use a reliability sample size method for percentiles. Tail estimates such as B10 usually need more data than estimates near the center of the distribution.
Benefits of increasing sample size
Increasing sample size has clear benefits when the added observations are representative and measured well.
Larger samples can:
Reduce the width of confidence intervals
Improve the chance of detecting meaningful differences
Help reveal shifts, mixtures, and special causes
Provide better estimates of standard deviation
Improve capability analysis
Build confidence in control plan decisions
Reduce the influence of unusual observations
The benefit is not linear. To cut the margin of error in half, the sample size must increase by about four times, assuming the same variation. That relationship comes from the square root effect in many sample size formulas.
For example, with the soup can process, increasing the sample from 50 to 200 does not make the estimate four times as precise. It makes the margin of error about half as large. That may be worthwhile for a high-volume product where tiny overfill reductions save significant material. It may be excessive for a quick screening study.
More data also cannot fix poor sampling. A large biased sample can be worse than a smaller well-designed sample because it creates false confidence.
A practical checklist for Lean Six Sigma teams
Before finalizing a sample size, work through these questions:
What exact decision will the sample support?
Is the response continuous, attribute, or time-to-failure?
Are we estimating a value or comparing groups?
What confidence level and power are required?
What margin of error or practical difference matters?
What standard deviation or baseline rate should we use?
Is the data likely to be non-normal, skewed, mixed, or censored?
Does the sampling plan cover normal production variation?
Is the measurement system capable?
10. Do we need extra samples for scrap, censoring, or invalid results?
This checklist keeps the calculation connected to the process. It also prevents the common mistake of treating sample size as a software output rather than a project decision.
The key takeaway
Sample size is not a statistical formality. It shapes the reliability of every Lean Six Sigma conclusion drawn from production data.
For soup can fill weights, a straightforward mean-based calculation may be enough if the process is stable and the data is reasonably well behaved. For bearing life, skewness and censoring often require log transformations or reliability methods. In both cases, the same principle applies: define the question, estimate variation, set the acceptable risk, and collect data that represents the process.
A well-planned sample gives the team a stronger basis for action. It reduces guesswork, protects against false improvement claims, and helps quality decisions stand up under real production conditions.
Comments