JOURNAL OF RESEARCH IN NATIONAL DEVELOPMENT VOLUME 8 NO 2, DECEMBER, 2010
DESIGN AND ANALYSIS OF
FACTORIAL
EXPERIMENTS OF VARIABLES AFFECTING STUDENTS’
ACADEMIC PERFORMANCE
Yakubu Yisa
Department of Mathematics and Statistics, Federal
University of Technology, Minna, Nigeria
Email:yisa_yakubu@yahoo.com
Abstract
In this paper, a
factorial
experiment is designed to examine the influence of
such factors as teaching method, gender and level
of study on students’ academic performance. The
subjects were tested on effect of teaching method,
gender and level of study on their academic
performance. The data obtained from the experiment
were analyzed using the Analysis of Variance
technique of
factorial
designs devised by Yates (1937), known as Yates
Algorithm. In the analysis process, the magnitude
and direction of the factor effects were first
examined to determine the likely important
variables. It was found that each of the Level,
Method, and LevelMethod interaction effects has
large impact on students’ academic performance
while Gender and all the other interaction effects
do not appear to have impact on students’
performance. The significance of these effects
with large impact was then confirmed by the
analysis of variance, which shows that teaching
method, level of study, and levelmethod
interaction, have significant effects on students’
academic performance, at 5% level of significance,
while gender and each of the other interactions
have no effect on students’ performance.
Keywords:
Factorial experiments, academic performance,
algorithm, interaction.
Introduction
The quality of a nation’s education system
directly affects its economy through the quality
of the workforce available to employers, the
demographics of the consumer market, the
productive capability of the economy, the pace of
innovation, and the relative standing of the
nation globally. The quality of education
determines the life opportunities available to
individuals and their ability to exercise their
right to life, liberty and the pursuit of
happiness. For a nation’s education system to
attain high quality level, timely assessments and
review of the methods by which students are taught
in schools, among other factors, is very vital.
This paper is on an experiment that seeks to
examine the role of teaching methods in schools,
level of study and gender, on academic
performances of students. The paper investigates
the influence of teaching method, gender and level
of study on students’ academic performance using a
factorial
design.
Design of experiment is the design of all
informationgathering exercises where variation is
present, whether under the full control of the
experimenter or not. When several factors are of
interest in an experiment, a factorial
experimental design is the most efficient. A
factorial
experiment is an experiment whose design
consists of two or more factors, each with
discrete possible values or "levels", and whose
experimental units take on all possible
combinations of these levels across all such
factors. The design technique involves varying
several factors simultaneously and drawing out the
individual effect of the factors and looking for
any possible combination (interaction) effects.
Factorial designs are frequently used in
experiments involving several factors where it is
necessary to investigate the joint effect of the
factors on a response (Montgomery, 1996). These
experiments enable us to investigate all possible
combinations of the levels of the factors in each
complete replication. The
type
of these designs is used here with k = 3. This
type of design provides the smallest number of
runs for which
k
factors can be studied in a complete factorial
design.
The teaching method consists of two levels which
include the traditional lecture method and the
Computer Assisted Learning method. Computer
Assisted Learning (CAL) is a computer program or
file developed specifically for educational
purposes.
This study examines the importance of CAL, among
the level
and gender
factors, to students of tertiary institutions. A
12year metaanalysis of research by the U.S.
Department of Education found that higher
education students in online learning generally
performed better than those in facetoface
courses (Means et al, 2009).
The popularization of this form of learning and
the increased ownership of personal computers led
to the development of widely distributed
educational CDROMS such as Encarta (Roschelle et
al, 2005).
Information technology (IT) is an enabler and
facilitator of
human capability. As technology continued to grow
and with the introduction of the Internet,
information on CAL programs became more
interactive, reflecting a social need for flexible
learning outcomes.
Factorial designs are more efficient than one
factor at a time experiments. The experiment
allows for estimation of experimental error in two
ways: the experiment can be replicated, or the
sparsity –of –effects principle can often be
exploited. These designs enable us to examine any
possible interactions to avoid misleading
conclusions. Factorial designs also allow us to
estimate effects of a factor at several levels of
the other factors, thereby yielding conclusions
that are valid over a range of experimental
conditions.
Methodology
Data collection
The experiment involves randomly selected national
diploma students (96), part – time programme, from
the business studies department of Niger State
Polytechnic, Zungeru, Bida campus. Forty eight
(48) of these students consisting of twenty four
(24) males and twenty four (24) females, all
selected randomly, were from ND1 and the same
number from ND2. The selected students from each
level were then randomly subdivided into two
groups of 24 each, with each group made up of 12
males and 12 females. We wish to stress here that
from the beginning of the experiment, all the
selections and groupings were done randomly using
the technique of simple random sampling, to ensure
experimental law and order. Students from the
first group of each level were then taught a Cost
Accounting course with the traditional lecture
method for two months while those in the second
group received the same lecture with the CAL
package, which gives full tutorial in modules, for
the same duration. Thus in each level, the first
group is the
Control
while the second group is the
Experimental group. At the end of the period,
a test was conducted separately for each of the
two groups in each level and the obtained scores
recorded. Thus the three factors in this design
are the level (ND1 and ND2), the lecture method
(Traditional and CAL, i.e., the Control and
Experimental) and the gender (Male and Female).
Analysis
The method of analysis used to analyze the
generated data (test scores) is the Analysis of
Variance (ANOVA) technique of
factorial
designs devised by Yates (1937), known as Yates
Algorithm. The analysis will examine the
significance of each of the three main effects,
and each of all the possible interactions, on the
academic performance of the students. The layout
of the Yates Algorithm for the
factorial
experiments is given below:
Treatment combination 
Response

(1) 
(2) 
(3) 
Effect

Effect estimate (3)÷

Sum of squares
÷

(1) 




I 


A 




A 


B 




B 


Ab 




AB 


C 




C 


Ac 




AC 


Bc 




BC 


Abc 




ABC 


The total sum of squares is obtained using the
usual formula
The error sum of squares may be found by
The sum of squares of any effect is given by
Where
,
and the sign in the parenthesis is negative if the
factor is included in the effect and positive
otherwise; k is the number of factors and n is the
number of replicates.
Presentation of data
The summary of
the results obtained are shown in the tables
below:
Table
1: The
performance Data Set
Factor Levels 
Scores out of 20 obtained by twelve students 


14 
6 
9 
6 
12 
6 
12 
9 
8 
5 
12 
10 

9 
10 
8 
7 
10 
12 
12 
6 
8 
7 
13 
8 

9 
9 
17 
8 
13 
14 
16 
12 
15 
10 
18 
12 

20 
18 
21 
16 
20 
18 
13 
15 
14 
11 
17 
19 

16 
15 
10 
8 
5 
8 
5 
11 
6 
7 
10 
13 

11 
14 
15 
8 
10 
12 
5 
12 
10 
11 
9 
9 

17 
13 
11 
14 
11 
12 
11 
14 
15 
10 
16 
13 

14 
18 
20 
11 
10 
18 
21 
10 
20 
16 
9 
16 
Where
= level of study,
= ND1,
= ND2;
= Method by which
the student was taught,
= Traditional lecture method
(i.e., Control),
= CAL method (i.e., Experimental);
= Sex,
= female,
= male.
Table 2: The design outlay
M
S
14,6,9,6,12,6,12,9
16,15,10,8,5,8,5,11,6,7
9,9,17,8,13,14,16
17,13,11,14,11
8,5,12,10
10,13
12,15,10,18,12
12,11,14,15,10
(109)
(114)
(153)
16,13
(157)
L
9,10,8,7,10,12,12
11,14,15,8,10,12,5,12,10
20,18,21,16,20,18
14,18,20,11,10
6,8,7,13,8
(110)
11,9,9 (126)
13,15,14,11,17,19
18,21,10,20,16
(202)
9,16
(183)

Note: The numbers in brackets are the
totals in each sublevel.
The statistical model for the
design is given by
Where
is
the observed response when factor
A is at
the ith
level, B is at the jth
level, and C
is at the kth level for the
lth
replicate.
is
the overall mean effect,
is
the effect of the ith level of factor A,
is
the effect of the jth level of factor B, and
is
the effect of the kth level of factor C.
is
the effect of the interaction between
and
,
is
the effect of the interaction between
and
,
is
the effect of the interaction between
and
,
is the effect
of the interaction between
,
and
, and
is
a random error component.
The Yates algorithm table for the above data is as
given below:
Treatment combination 
Response

(1) 
(2) 
(3) 
Effect

Effect estimate (3)÷

Sum of squares
÷

(1) 
109 
219 
574 
1154 
I 
 
 
L 
110 
355 
580 
88 
L 
1.833 
80.667 
M 
153 
240 
50 
236 
M 
4.917 
580.167 
Lm 
202 
340 
38 
62 
LM 
1.292 
40.042 
S 
114 
1 
136 
6 
S 
0.125 
0.375 
Ls 
126 
49 
100 
12 
LS 
0.25 
1.500 
Ms 
157 
12 
48 
36 
MS 
0.75 
13.500 
Lms 
183 
26 
14 
34 
LMS 
0.708 
12.042 
From the effects estimate column of the algorithm
above, we see that the largest effects are for
level (L = 1.833), method (M = 4.917), level –
method interaction (LM = 1.292), and gender (S =
0.125), which are all positive. These suggest that
students’ academic performance will improve when
(i)
they
proceed from low level of study (ND1) to the high
level (ND2);
(ii)
there
is a switch from the traditional lecture method
(Control) to the CAL method (Experimental).
The gender effect, though positive, does not
appear to have as large an impact on students’
performance as the Level and Method effects. All
the other effects are negative.
The analysis of variance further confirms the
significance of these positive effects and the
table is given below:
Anova table
Source of variation 
df 
SS 
MS 


L 
1 
80.667 
80.667 
8.144 
4.00 
M 
1 
580.167 
580.167 
58.573 
4.00 
Lm 
1 
40.042 
40.042 
4.043 
4.00 
S 
1 
0.375 
0.375 
0.038^{ns}

4.00 
Ls 
1 
1.500 
1.500 
0.151^{ns} 
4.00 
Ms 
1 
13.500 
13.500 
1.363^{ns} 
4.00 
Lms 
1 
12.042 
12.042 
1.216^{ns} 
4.00 
Error

88 
871.665 
9.905 


Total 
95 
1599.958 



^{ ns= not significant }
From the ANOVA table above, it can be concluded
that students’ academic performance depends on
level of study, method of teaching and the
interaction of level and method. Gender and all
other interactions do not have effect on students’
performance.
The profile plots of the three main effects and
the levelmethod interaction are given below for
comparing marginal means.

_
__
These
plots also indicate the positive effects of all
the three variables and the levelmethod
interaction. That is, moving from low level to the
high level moves the performance upward, but the
largest effect is that of the Method factor as can
be observed from the estimated marginal means axis
of the plots.
Conclusion
Factorial experimental designs and analysis are
powerful statistical techniques for more
complicated and realistic experiments involving
certain phenomena. From the analysis of the
designed
factorial
experiment involving level of study, method of
teaching and gender, it is observed that students’
academic performance is significantly influenced
by their level of study, method of teaching and
level – method interaction. A careful observation
of the profile plots shows that the computer
–aided learning (CAL) method of teaching has the
largest influence on students’ performance
followed by the second level of study. Therefore,
whilst Computer Assisted Learning may encounter
some negativity from people resistant to change,
there is no doubt that this educational tool is
extremely valuable. From children to adults, there
is much to be gained from CAL’s interactive and
self motivating format for learning.
References
Henry,
J. (1994): ‘Resistance to computerbased
technology in the workplace,’ in
Executive Development, vol. 7, no. 1, pp 20 –
23.
Means,
B. et al, (2009): ‘Evaluation of Evidence – Based
Practices in Online Learning: A
MetaAnalysis and Review of Online Learning
Studies,
http://www.ed.gov/rschstat/eval/tech/evidencebasedpractices/finalreport.pdf,
retrieved 20 August, 2009.
Montgomery, D. C. (1991):
Design and
Analysis of Statistical Experiments, New York:
John Wiley & Sons.
Roschelle, J. et al, (2005): ‘Introduction to the
special issue on wireless and mobile
technologies in education,’ in Journal of
Computer Assisted Learning, vol. 21, no.
3, pp
159 – 161.