advert

JOURNAL OF RESEARCH IN NATIONAL DEVELOPMENT VOLUME 8 NO 2, DECEMBER, 2010


 

 

DESIGN AND ANALYSIS OF  FACTORIAL EXPERIMENTS OF VARIABLES AFFECTING STUDENTS’ ACADEMIC PERFORMANCE

 

Yakubu Yisa

Department of Mathematics and Statistics, Federal University of Technology, Minna, Nigeria

E-mail:yisa_yakubu@yahoo.com

 


Abstract

In this paper, a  factorial experiment is designed to examine the influence of such factors as teaching method, gender and level of study on students’ academic performance. The subjects were tested on effect of teaching method, gender and level of study on their academic performance. The data obtained from the experiment were analyzed using the Analysis of Variance technique of  factorial designs devised by Yates (1937), known as Yates Algorithm. In the analysis process, the magnitude and direction of the factor effects were first examined to determine the likely important variables. It was found that each of the Level, Method, and Level-Method interaction effects has large impact on students’ academic performance while Gender and all the other interaction effects do not appear to have impact on students’ performance. The significance of these effects with large impact was then confirmed by the analysis of variance, which shows that teaching method, level of study, and level-method interaction, have significant effects on students’ academic performance, at 5% level of significance, while gender and each of the other interactions have no effect on students’ performance.

 

Keywords: Factorial experiments, academic performance, algorithm, interaction.


 

 Introduction

The quality of a nation’s education system directly affects its economy through the quality of the workforce available to employers, the demographics of the consumer market, the productive capability of the economy, the pace of innovation, and the relative standing of the nation globally. The quality of education determines the life opportunities available to individuals and their ability to exercise their right to life, liberty and the pursuit of happiness. For a nation’s education system to attain high quality level, timely assessments and review of the methods by which students are taught in schools, among other factors, is very vital. This paper is on an experiment that seeks to examine the role of teaching methods in schools, level of study and gender, on academic performances of students. The paper investigates the influence of teaching method, gender and level of study on students’ academic performance using a  factorial design.

 

Design of experiment is the design of all information-gathering exercises where variation is present, whether under the full control of the experimenter or not. When several factors are of interest in an experiment, a factorial experimental design is the most efficient. A factorial experiment is an experiment whose design consists of two or more factors, each with discrete possible values or "levels", and whose experimental units take on all possible combinations of these levels across all such factors. The design technique involves varying several factors simultaneously and drawing out the individual effect of the factors and looking for any possible combination (interaction) effects.

 

Factorial designs are frequently used in experiments involving several factors where it is necessary to investigate the joint effect of the factors on a response (Montgomery, 1996). These experiments enable us to investigate all possible combinations of the levels of the factors in each complete replication. The  type of these designs is used here with k = 3. This type of design provides the smallest number of runs for which k factors can be studied in a complete factorial design.

 

The teaching method consists of two levels which include the traditional lecture method and the Computer Assisted Learning method. Computer Assisted Learning (CAL) is a computer program or file developed specifically for educational purposes.

 

This study examines the importance of CAL, among the level and gender factors, to students of tertiary institutions. A 12-year meta-analysis of research by the U.S. Department of Education found that higher education students in online learning generally performed better than those in face-to-face courses (Means et al, 2009).

The popularization of this form of learning and the increased ownership of personal computers led to the development of widely distributed educational CDROMS such as Encarta (Roschelle et al, 2005).

 

Information technology (IT) is an enabler and facilitator of human capability. As technology continued to grow and with the introduction of the Internet, information on CAL programs became more interactive, reflecting a social need for flexible learning outcomes.

 

Factorial designs are more efficient than one factor at a time experiments. The experiment allows for estimation of experimental error in two ways: the experiment can be replicated, or the sparsity –of –effects principle can often be exploited. These designs enable us to examine any possible interactions to avoid misleading conclusions. Factorial designs also allow us to estimate effects of a factor at several levels of the other factors, thereby yielding conclusions that are valid over a range of experimental conditions.   

 

 Methodology

Data collection

The experiment involves randomly selected national diploma students (96), part – time programme, from the business studies department of Niger State Polytechnic, Zungeru, Bida campus. Forty eight (48) of these students consisting of twenty four (24) males and twenty four (24) females, all selected randomly, were from ND1 and the same number from ND2. The selected students from each level were then randomly subdivided into two groups of 24 each, with each group made up of 12 males and 12 females. We wish to stress here that from the beginning of the experiment, all the selections and groupings were done randomly using the technique of simple random sampling, to ensure experimental law and order. Students from the first group of each level were then taught a Cost Accounting course with the traditional lecture method for two months while those in the second group received the same lecture with the CAL package, which gives full tutorial in modules, for the same duration. Thus in each level, the first group is the Control while the second group is the Experimental group. At the end of the period, a test was conducted separately for each of the two groups in each level and the obtained scores recorded. Thus the three factors in this design are the level (ND1 and ND2), the lecture method (Traditional and CAL, i.e., the Control and Experimental) and the gender (Male and Female).

 

Analysis

The method of analysis used to analyze the generated data (test scores) is the Analysis of Variance (ANOVA) technique of  factorial designs devised by Yates (1937), known as Yates Algorithm. The analysis will examine the significance of each of the three main effects, and each of all the possible interactions, on the academic performance of the students. The layout of the Yates Algorithm for the  factorial experiments is given below:


 

Treatment combination

Response

(1)

(2)

(3)

Effect

Effect estimate (3)÷

Sum of squares ÷

(1)

 

 

 

 

I

 

 

A

 

 

 

 

A

 

 

B

 

 

 

 

B

 

 

Ab

 

 

 

 

AB

 

 

C

 

 

 

 

C

 

 

Ac

 

 

 

 

AC

 

 

Bc

 

 

 

 

BC

 

 

Abc

 

 

 

 

ABC

 

 

 

 


The total sum of squares is obtained using the usual formula

The error sum of squares may be found by

The sum of squares of any effect is given by

Where

  ,

and the sign in the parenthesis is negative if the factor is included in the effect and positive otherwise; k is the number of factors and n is the number of replicates.

 

Presentation of data

The summary of the results obtained are shown in the tables below:-

 


 

 

 

 

Table 1: The performance Data Set

Factor Levels

Scores out of 20 obtained by twelve students

      

14

6

9

6

12

6

12

9

8

5

12

10

      

9

10

8

7

10

12

12

6

8

7

13

8

      

9

9

17

8

13

14

16

12

15

10

18

12

      

20

18

21

16

20

18

13

15

14

11

17

19

      

16

15

10

8

5

8

5

11

6

7

10

13

      

11

14

15

8

10

12

5

12

10

11

9

9

      

17

13

11

14

11

12

11

14

15

10

16

13

      

14

18

20

11

10

18

21

10

20

16

9

16

 

Where

            = level of study, = ND1, = ND2;

= Method by which the student was taught,  = Traditional lecture method                                                                          (i.e., Control), = CAL method (i.e., Experimental);

            = Sex, = female, = male.

 

 

 

Table 2: The design outlay

M

                                                                                                                   

                     

S

                                                                                                                    

    

          14,6,9,6,12,6,12,9   16,15,10,8,5,8,5,11,6,7          9,9,17,8,13,14,16     17,13,11,14,11

               8,5,12,10                 10,13                                      12,15,10,18,12         12,11,14,15,10

                     (109)                          (114)                                      (153)                   16,13      (157)

L                                                                                                                               

 

         9,10,8,7,10,12,12    11,14,15,8,10,12,5,12,10          20,18,21,16,20,18   14,18,20,11,10

               6,8,7,13,8 (110)      11,9,9 (126)                            13,15,14,11,17,19     18,21,10,20,16

                                                                                                   (202)                      9,16 (183)

 

 

 


Note: The numbers in brackets are the totals in each sub-level.

The statistical model for the design is given by

Where  is the observed response when factor A is at the ith level, B is at the jth level, and C is at the kth level for the lth replicate.  is the overall mean effect,  is the effect of the ith level of factor A,  is the effect of the jth level of factor B, and  is the effect of the kth level of factor C.  is the effect of the interaction between  and ,  is the effect of the interaction between  and ,  is the effect of the interaction between  and ,  is the effect of the interaction between ,  and , and is a random error component.


 

 

The Yates algorithm table for the above data is as given below:

 

Treatment combination

Response

(1)

(2)

(3)

Effect

Effect estimate (3)÷

Sum of squares ÷

(1)

109

219

574

1154

I

-

-

L

110

355

580

88

L

1.833

80.667

M

153

240

50

236

M

4.917

580.167

Lm

202

340

38

62

LM

1.292

40.042

S

114

1

136

6

S

0.125

0.375

Ls

126

49

100

-12

LS

-0.25

1.500

Ms

157

12

48

-36

MS

-0.75

13.500

Lms

183

26

14

-34

LMS

-0.708

12.042

 


From the effects estimate column of the algorithm above, we see that the largest effects are for level (L = 1.833), method (M = 4.917), level – method interaction (LM = 1.292), and gender (S = 0.125), which are all positive. These suggest that students’ academic performance will improve when

(i)                 they proceed from low level of study (ND1) to the high level (ND2);

(ii)               there is a switch from the traditional lecture method (Control) to the CAL method (Experimental).

The gender effect, though positive, does not appear to have as large an impact on students’ performance as the Level and Method effects. All the other effects are negative.

The analysis of variance further confirms the significance of these positive effects and the table is given below:


 

 

Anova table

Source of variation

df 

SS

MS

 

 

L

1

80.667

80.667

8.144

4.00

M

1

580.167

580.167

58.573

4.00

Lm

1

40.042

40.042

4.043

4.00

S

1

0.375

0.375

0.038ns

4.00

Ls

1

1.500

1.500

0.151ns

4.00

Ms

1

13.500

13.500

1.363ns

4.00

Lms

1

12.042

12.042

1.216ns

4.00

Error

88

871.665

9.905

 

 

Total

95

1599.958

 

 

 

ns= not significant

 


From the ANOVA table above, it can be concluded that students’ academic performance depends on level of study, method of teaching and the interaction of level and method. Gender and all other interactions do not have effect on students’ performance.

The profile plots of the three main effects and the level-method interaction are given below for comparing marginal means.


 

 

 

----

 

_ __

 

 


 

These plots also indicate the positive effects of all the three variables and the level-method interaction. That is, moving from low level to the high level moves the performance upward, but the largest effect is that of the Method factor as can be observed from the estimated marginal means axis of the plots.

Conclusion

Factorial experimental designs and analysis are powerful statistical techniques for more complicated and realistic experiments involving certain phenomena. From the analysis of the designed  factorial experiment involving level of study, method of teaching and gender, it is observed that students’ academic performance is significantly influenced by their level of study, method of teaching and level – method interaction. A careful observation of the profile plots shows that the computer –aided learning (CAL) method of teaching has the largest influence on students’ performance followed by the second level of study. Therefore, whilst Computer Assisted Learning may encounter some negativity from people resistant to change, there is no doubt that this educational tool is extremely valuable. From children to adults, there is much to be gained from CAL’s interactive and self motivating format for learning.

 

References

Henry, J. (1994): ‘Resistance to computer-based technology in the workplace,’ in     Executive Development, vol. 7, no. 1, pp 20 – 23.

Means, B. et al, (2009): ‘Evaluation of Evidence – Based Practices in Online Learning: A     Meta-Analysis and Review of Online Learning Studies,     http://www.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf,             retrieved 20 August, 2009.

Montgomery, D. C. (1991): Design and Analysis of Statistical Experiments, New York: John Wiley & Sons.

Roschelle, J. et al, (2005): ‘Introduction to the special issue on wireless and mobile   technologies in education,’ in Journal of Computer Assisted Learning, vol. 21, no.   3, pp 159 – 161.