Standard Deviation


Definition/Introduction

The standard deviation (SD) measures the extent of scattering in a set of values, typically compared to the mean value of the set.[1][2][3] The calculation of the SD depends on whether the dataset is a sample or the entire population. Ideally, studies would obtain data from the entire target population, which defines the population parameter. However, this is rarely possible in medical research, and hence a sample of the population is often used.[4]

The sample SD is determined through the following steps:

  1. Calculate the deviation of each observation from the mean
  2. Square each of these results
  3. Add these results together
  4. Divide this sum by the 'total number of observations minus 1' - the result at this stage is called the sample variance.
  5. Square root this result - This number will be the sample SD.[1]

For example, to work out the sample SD of the following data set: (4, 5, 5, 5, 7, 8, 8, 8, 9, 10)

The first step would be to calculate the mean of the data set. This is done by adding the value of each observation together and then dividing by the number of observations. The sum of the values would be 69, which then is divided by 10, so the mean would be 6.9. 

Then we would follow the same steps as above to work out the standard deviation:

  1. To calculate the deviation, subtract the mean from every observation which would result in the following values: (-2.9, -1.9, -1.9, -1.9, 0.1, 1.1, 1.1, 1.1, 2.1, 3.1)
  2. Then each value is squared to remove any negative values, resulting in the following values: (8.41, 3.61, 3.61, 3.61, 0.01, 1.21, 1.21, 1.21, 4.41, 9.61)
  3. The sum of these values is then calculated, which is 36.9
  4. The sample variance is then calculated. This is done by dividing the current value by the 'total number of observations minus 1', which in this case is 9: 36.9/9 = 4.1.
  5. Finally, the result is square rooted to find the sample standard deviation, which is 2.02 (to two decimal places).

The population SD is calculated similarly, with the only difference being in 'step 4' divided by the 'total number of observations' instead of ‘total number of observations minus 1’.

For example, if we take the same data set used above but instead calculate the SD assuming the data set was the total population.

  1. To calculate the deviation, subtract the mean from every observation which would result in the following values: (-2.9, -1.9, -1.9, -1.9, 0.1, 1.1, 1.1, 1.1, 2.1, 3.1)
  2. Then each value is squared to remove any negative values, resulting in the following values: (8.41, 3.61, 3.61, 3.61, 0.01, 1.21, 1.21, 1.21, 4.41, 9.61)
  3. The sum of these values is then calculated, which is 36.9
  4. The population variance is then calculated. This is done by dividing the current value by the 'total number of observations,' which in this case is 10. 36.9/10 = 3.69
  5. Finally, the result is square rooted to find the population standard deviation, which is 1.92 (to two decimal places)

As can be seen, by the formulas above, a large SD results from data with a large deviation from the mean, while a small SD shows the values are all quite close to the mean.[2][5]

Issues of Concern

Although the abovementioned method of calculating sample SD will always result in the correct value, as the data set increases in quantity, this method becomes increasingly burdensome. This is mainly due to having to calculate the difference from the mean of every value. This step can be bypassed through the following method:

  1. List the observed values of the data set in a column which we shall call 'x.'
  2. Find the 'sum of x' by adding all of the values in the column together.
  3. Square the value of the 'sum of x.'
  4. Divide this result by the 'total number of observations.' Call the result of this division 'y'.
  5. Square each value within the column 'x.'
  6. Find the sum of the above squares.
  7. Subtract the value 'y' from this sum.
  8. Divide this result by the 'total number of observations minus one' if calculating the sample SD. Divide this result by the 'total number of observations' if calculating the population SD.
  9. Finally, the result is square rooted to calculate the SD.

This method can quickly calculate the sample SD of a large data set, especially with a calculator with a memory function or an electronic data analysis program.

A mistake sometimes seen in research papers is whether the SD or the standard error of the mean (SEM) should be reported alongside the mean. The distinction between the SD and SEM is crucial but often overlooked. This results in authors reporting the incorrect one alongside their data. While the SD refers to the scatter of values around the sample mean, the SEM refers to the accuracy of the sample mean itself. This means the role of the SEM is to provide a measurement of the precision of the sample mean compared to the total population mean. Consequently, in contrast to the SD, the SEM does not provide information on the scatter of the sample.[5]

Despite this difference, the SEM is still often used in places where the SD should be stated. There have been many reasons hypothesized for this, such as a lack of full understanding of the meaning of these statistical concepts, leading authors to report what they have seen other authors report in their studies. This leads to multiple articles all publishing the SEM in an improper context. Another reason is that the SEM will be smaller than the SD and hence if presented alongside the mean, can give the impression that the data is more precise.

This incorrect impression would also be seen in the figures, as using the SEM will shorten the error bars. This can confuse even with experienced readers as the reader is expecting the SD to be paired with the mean and hence may incorrectly believe that the quoted SEM refers to the scatter of values around the mean (the SD).[5][6] In response to these issues, some journals only allow authors to present SDs and not SEMs to remove any chance of an author mistakenly using the SEM in an inappropriate context.[6][7]

Clinical Significance

Using SD allows for a quick overview of a population as long as the population follows a normal (Gaussian) distribution. In these populations, we know that 1 SD covers 68% of observations, 2 SD covers 95% of observations, and 3 SD covers 99.7% of observations.[5][6][8][9] This adds greater context to the mean value that is stated in many studies.

A hypothetical example will help clarify this point: Say a study looking at the effect of a new chemotherapy agent on life expectancy concludes that it increases life expectancy by 5 years.

A mean of 5 years could be achieved by widely different data sets. For example, everyone in the sample could have shown an increase between 4 to 6 years in life expectancy. However, another data set that would satisfy this mean would be if half the sample showed no statistical increase in life expectancy while the other half had an increase of 10 years. Including an SD can help readers quickly resolve this ambiguity as the former case would have a small SD while the latter would have a large SD. This allows readers to interpret the results of the study more accurately.

Furthermore, finding a “normal” range of measurements within medicine is often a key piece of information. One example would be the range of a laboratory value (such as full blood count) expected if the measurement was conducted in healthy individuals. Often this is the 95% reference range, where 2.5% of values will be lesser than the reference range, and 2.5% of values will be greater than the reference range. This makes the SD extremely useful in data sets that follow a normal distribution because it can quickly calculate the range in which 95% of values lie.[10]

Another advantage of reporting the SD is that it reports the scatter within the data in the same units as the data itself. This is in contrast to the variance of the data set, which is equivalent to the square of the SD, and hence its units are the units of the data squared. Therefore, although the variance can be useful in certain scenarios, it is generally not used in data description. Through sharing the same unit, the SD allows for the data to be more easily interpreted.[1][2]

It is also important to know the disadvantages of using an SD. The main issue is in data sets where there are extreme values or severe skewness, as these results can influence the mean and SD by a significant amount. Consequently, in scenarios where the data set does not follow a normal (Gaussian) distribution, other measures of dispersion are often used. Most commonly, the interquartile range (IQR) is used alongside the median of the dataset. This is due to the IQR being significantly more resistant to extreme values, as when calculating the IQR, only the data between the first and third quartiles are factored in. The data between the first and third quartile represents the middle 50% of values, so any unusually high or low values will not affect the calculation of the IQR. This gives a more accurate picture of the data set’s distribution than the SD.[1][11][12][13]

Nursing, Allied Health, and Interprofessional Team Interventions

Evidence-based medicine plays a core role in patient care. It relies on integrating clinical expertise alongside the current best available literature.[14] Consequently, keeping up to date with the current literature is key for all members of an interprofessional team to ensure they are acting in the best interest of a patient. To do this, all members must have a basic understanding of descriptive statistics (including the mean, median, mode, and standard deviation). Without this knowledge, it can be difficult to interpret the conclusions of research articles accurately.


Details

Author

Samy El Omda

Updated:

8/14/2023 9:32:31 PM

References


[1]

Whitley E, Ball J. Statistics review 1: presenting and summarising data. Critical care (London, England). 2002 Feb:6(1):66-71     [PubMed PMID: 11940268]


[2]

Vetter TR. Descriptive Statistics: Reporting the Answers to the 5 Basic Questions of Who, What, Why, When, Where, and a Sixth, So What? Anesthesia and analgesia. 2017 Nov:125(5):1797-1802. doi: 10.1213/ANE.0000000000002471. Epub     [PubMed PMID: 28891910]


[3]

Wissing DR, Timm D. Statistics for the nonstatistician: Part I. Southern medical journal. 2012 Mar:105(3):126-30. doi: 10.1097/SMJ.0b013e3182498ad5. Epub     [PubMed PMID: 22392207]


[4]

Vetter TR. Fundamentals of Research Data and Variables: The Devil Is in the Details. Anesthesia and analgesia. 2017 Oct:125(4):1375-1380. doi: 10.1213/ANE.0000000000002370. Epub     [PubMed PMID: 28787341]


[5]

Andrade C. Understanding the Difference Between Standard Deviation and Standard Error of the Mean, and Knowing When to Use Which. Indian journal of psychological medicine. 2020 Jul:42(4):409-410. doi: 10.1177/0253717620933419. Epub 2020 Jul 20     [PubMed PMID: 33402813]

Level 3 (low-level) evidence

[6]

Barde MP, Barde PJ. What to use to express the variability of data: Standard deviation or standard error of mean? Perspectives in clinical research. 2012 Jul:3(3):113-6. doi: 10.4103/2229-3485.100662. Epub     [PubMed PMID: 23125963]

Level 3 (low-level) evidence

[7]

Bartko JJ. Rationale for reporting standard deviations rather than standard errors of the mean. The American journal of psychiatry. 1985 Sep:142(9):1060     [PubMed PMID: 4025621]


[8]

Swinscow TD. Statistics at square one. III--Standard deviation. British medical journal. 1976 Jun 5:1(6022):1393-4     [PubMed PMID: 1276702]


[9]

Lee DK, In J, Lee S. Standard deviation and standard error of the mean. Korean journal of anesthesiology. 2015 Jun:68(3):220-3. doi: 10.4097/kjae.2015.68.3.220. Epub 2015 May 28     [PubMed PMID: 26045923]


[10]

Hazra A, Gogtay N. Biostatistics Series Module 7: The Statistics of Diagnostic Tests. Indian journal of dermatology. 2017 Jan-Feb:62(1):18-24. doi: 10.4103/0019-5154.198047. Epub     [PubMed PMID: 28216720]


[11]

Driscoll P, Lecky F, Crosby M. An introduction to everyday statistics--2. Journal of accident & emergency medicine. 2000 Jul:17(4):274-81     [PubMed PMID: 10921817]


[12]

Hohmann E, Wetzler MJ, D'Agostino RB Jr. Research Pearls: The Significance of Statistics and Perils of Pooling. Part 2: Predictive Modeling. Arthroscopy : the journal of arthroscopic & related surgery : official publication of the Arthroscopy Association of North America and the International Arthroscopy Association. 2017 Jul:33(7):1423-1432. doi: 10.1016/j.arthro.2017.01.054. Epub 2017 Apr 28     [PubMed PMID: 28457678]


[13]

Madadizadeh F, Ezati Asar M, Hosseini M. Common Statistical Mistakes in Descriptive Statistics Reports of Normal and Non-Normal Variables in Biomedical Sciences Research. Iranian journal of public health. 2015 Nov:44(11):1557-8     [PubMed PMID: 26744717]


[14]

Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ (Clinical research ed.). 1996 Jan 13:312(7023):71-2     [PubMed PMID: 8555924]