Standard error and significance level

In order to know how accurate our single sample based regression coefficient is as an estimate of the population coefficient, we need to know the size of the standard error. Fortunately, although we cannot find its exact value, we can get a fairly accurate estimate of it through analysis of our sample data. This estimate, which is reported in the SPSS regression analysis coefficients table, makes it possible to tell how likely it is that the difference between the population regression coefficient and our sample regression coefficient is larger or smaller than a certain, freely chosen value. This makes it possible to test so called null hypotheses about the value of the population regression coefficient.

Such testing is easy with SPSS if we accept the presumption that the relevant null hypothesis to test is the hypothesis that the population has a zero regression coefficient, i.e. that there is no linear association between the independent and the dependent variable. Our test criterion will be that the null hypothesis shall be refuted if there is less than a certain likelihood (e.g. 5% likelihood) that a population with a coefficient value of 0 would give rise to a sample with a regression coefficient whose absolute value is equal to or larger than the one we actually found in our sample. We call this chosen likelihood level our ‘significance level’. Note that we cannot conclude with certainty whether or not the null hypothesis is true. This criterion says that we should refute the null hypothesis if the chances that we would observe the estimated regression coefficient if the null hypothesis really were true is less than our chosen significance level. Thus, if we choose 5 % likelihood as our criterion, there is a 5% chance that we might refute a correct null hypothesis. Refuting a correct null hypothesis is called a ‘type 1 error’. If we think that a 5% percentage chance of making such an error is too high, we should choose a smaller significance level, say a 1% level. The most common significance levels are 10%, 5% and 1%. Higher levels than 10% are very rare. Levels that are lower than 1% may occur. But note that choosing a low significance level and, hence, a low risk of committing a type 1 error, comes at the cost of choosing a high risk of committing a ‘type 2 error’, which is the error of omitting to refute an incorrect null hypothesis. But this risk decreases with the size of the sample, so, with large samples, one may prefer small significance levels.

Go to next page >>