|
8.1 Introduction to surveys
8.2 Methodological approaches
8.3 Doing survey research
8.4 Statistical Analysis
8.4.1 Descriptive statistics
8.4.2 Exploring relationships
8.4.3 Analysing samples
8.4.3.1 Generalising from samples
8.4.3.2 Dealing with sampling error
8.4.3.3 Confidence limits
8.4.3.4 Statistical significance
8.4.3.5 Hypothesis testing
8.4.3.6 Significance tests
8.4.3.6.1 Parametric tests of significance
8.4.3.6.2 Non-parametric tests of significance
8.4.3.6.2.1 Chi-square test
8.4.3.6.2.2 Mann Witney U test
8.4.3.6.2.3 Kolmogorov-Smirnov Test
8.4.3.6.2.4 H test
8.4.3.6.2.5 Sign test
8.4.3.6.2.5.1 A Note on the McNemar test
8.4.3.6.2.6 Wilcoxon test
8.4.3.6.2.7 Friedman test
8.4.3.6.2.8 Q test
8.4.3.7 Summary of significance testing and association: an example
8.4.4 Report writing
8.5 Summary and conclusion
When to use the Sign test
Hypotheses
What we need to know
Sampling distribution
Assumptions
Testing Statistic
Critical values
Worked examples
Unworked examples
When to use the Sign test
The Sign test is used to test to related samples. It is particularly useful for matched pairs (for example, a retested sample).
It requires ordinal (or higher level) data, although it can be used on bivariate nominal data (such as Blue/Red, Voter/Non-voter).
Top
Hypotheses
H0:The two related samples are from the same population
HA:The related samples are not from the same population, there is an unspecified direction change (two-tail test).
HA:The related samples are not from the same population, there is a change in a specified direction (one-tail test).
Top
What we need to know
The Sign test requires that the individual sample values are known for the related pairs.
In addition it is necessary to be able to allocate a 'positive' or 'negative' label to each change between related pairs. For ordinal data (or higher level) such labelling of change is straightforward.
If the data is biviriate nominal then one change (from red to blue, for example) must be designated 'negative' and the reverse designated 'positive'.
Positive changes are given a plus (+) sign and negative changes a minus (-) sign. Hence the name of the test.
Top
Sampling distribution
The Sign test is based on the null hypothesis that if the changes are random then the probability of a + sign is equal to the probability of a - sign, both being p=0.5.
Hence the binomial distribution can be applied to find out the probability of a given number of pluses (or minuses) in a sample. (See Engineering Statistics Handbook for a discussion of binomila distribution (accessed 24 May 2020))
From the binomial distribution, the average number of pluses in the sample is np, where n is the sample size and p is the probability of a plus (i.e. n/2 when p=0.5)
The variance of a binomial distribution is npq (where q = 1-p); in this case variance = n/4 or Standard deviation= √n/2 (as both p and q = 0.5)
The normal approximation to the binomial can be used for sample of 5 or more when p=0.5.
(Strictly speaking np>5 is required for the binomial to approximate a normal distribution, which requires a minimum sample size of 10. However, with p=0.5, the results for the normal approximation at 5% and 1% signifcance level are not affectd for sample between 5 and 10. The sign test is inapplicable for samples under five).
The main limitation of the test is that it ignores situations when there is 'no change' between samples. Any 'no change' data is discarded and the sample size is reduced. The Sign test is therefore not applicable where several 'no change' situations occur.
Top
Assumptions
There are no assumptions underlying the use of this test provided the change between related pairs can validly be divided into two categories.
(Although, theoretically, the variable being considered should have an underlying continuous distribution.)
Top
Testing statistic
As the normal approximation to the binomial is employed for all applicable sample sizes (viz. n>5) the testing statistic is a z statistic:
z = [(Number of minuses in the sign test) - (mean number of minuses (i.e.n/2))]/SD of number of minuses, i.e. square root of sample size(n) divided by 2
thus z = [2(number of minuses) - n]/√n
Top
Critical values
The critical values are the same for any z test (see See Section 8.4.3.5 for some typical critical values for the z test). For example critical z = 1.96 for a two-tail test at 95% confience level.
Top
Worked example
IQ test scores for nine pupils in a test-retest experiment are shown below
| Pupil |
Test 1 |
Test 2 |
Change |
| A |
100 |
99 |
- |
| B |
103 |
96 |
- |
| C |
104 |
92 |
- |
| D |
115 |
103 |
- |
| E |
100 |
104 |
+ |
| F |
117 |
118 |
+ |
| G |
111 |
100 |
- |
| H |
99 |
97 |
- |
| I |
102 |
100 |
- |
Hypotheses:
H0: There is no significant difference in the scores between the tests.
HA: There is a significant drop in the the scores.
This is a one-tail test of significance.
Significance level: 5%
Testing statistic: z
z = [2(number of minuses) - n]/√n
where n is the matched sample size (i.e. number of pairs excluding the ones with no change).
Critical value: From tables of critical z, 95% confidence level, one-tail test z=1.64
Decision rule: reject H0 if calculated z > 1.64
Computation:
Number of minuses = 7
n = 9
z = (2(7) - 9)/√9 = (14-9)/3 = 5/3 = 1.66
Decision: Reject the null hypotheses (H0) as calculated z is greater than the critical value of 1.64; there is a signifcant drop in scores.
Top
Unworked examples
Ten history students were asked to rate (out of 10) the degree to which Lenin understood the feelings of the Russian workers in 1917. They were asked to read John Reed's book Ten Days That Shook the World and then do the rating again. Is there a significant change in rating in the table below?
| Student |
Before |
After |
| A |
4 |
7 |
| B |
8 |
10 |
| C |
3 |
8 |
| D |
5 |
7 |
| E |
4 |
4 |
| F |
6 |
8 |
| G |
5 |
8 |
| H |
6 |
9 |
| I |
10 |
9 |
| J |
7 |
9 |
Top
2. A class of 20 senior school students took both English and History final-year examinations. The table below shows grades achieved by each student. Are the English grades significantly better than the History grades (A grade is highest, F grade is lowest).
| Student number |
English Grade |
History Grade |
| 1 |
A |
C |
| 2 |
B |
D |
| 3 |
F |
E |
| 4 |
B |
E |
| 5 |
C |
B |
| 6 |
D |
E |
| 7 |
A |
B |
| 8 |
A |
D |
| 9 |
B |
A |
| 10 |
C |
B |
| 11 |
C |
D |
| 12 |
E |
D |
| 13 |
B |
C |
| 14 |
A |
B |
| 15 |
A |
E |
| 16 |
A |
D |
| 17 |
B |
D |
| 18 |
C |
E |
| 19 |
B |
F |
| 20 |
D |
B |
Top
8.4.3.6.2.5.1 A Note on the McNemar test
This may be converted for continuity using Yate's correction. To use this test the expected frequency must be 5 or more, hence the sample size must be at least 10.
Consequently the test has no obvious advantages over the Sign test, a point reinforced by the fact that the power of the two tests is identical.
|
