Methodology
Sample
The research sample used in this study included 340students at a large mid-Atlantic university in the USA. The sample size is comparable to similar
studies on user satisfaction such as McHaney and Cronan (1998) and McHaney et al. (2002). Although
the sample for this study was collected at a higher education institution, it represents the end
user population well, because some students were full-time students while others were part-time
from a variety of industries and management levels.
Table 11-1
provides sample descriptive statistics. As the table indicates, the subjects are distributed evenly
in male (52%) and female (48%). As for work experience, about 75% of the subjects have a few years
of experience (35% + 41%), while 25% had more than three years of experience. Most of the subjects
use computers at home (86%) and school (77%), and close to one half (41%) of the subjects use
computers at work. Some subjects use computers at multiple places.
Gender | Male | Female | Total* | |||
174 (52%) | 158 (48%) | 332 | ||||
Work Experience | < 1 year | 1 - 3 years | 4 - 6 years | 7 - 10 years | > 10 years | |
115 (35%) | 134 (41%) | 52 (16%) | 14 (4%) | 14 (4%) | 329 | |
Location (multiple choices) | Home | Work | School | |||
291 (86%) | 138 (41%) | 261 (77%) | 690 | |||
* N = 340. There are some missing data. |
Data Collection
The data for this study were collected in 2002 as part of a larger research study on usersatisfaction with Web-based information systems. A survey instrument was pilot tested with 30
graduate students and the instrument proved to be valid and reliable. The research questionnaire
was then distributed in class to 340 graduate and undergraduate students. While collecting the
data, students were informed that participation in this study was voluntary and anonymous. No
personal identity information was collected in the survey. Hard copy questionnaires were
distributed in class by the researcher. It took about 10 minutes to complete the survey.
Analysis
Analysis of variance (ANOVA) was performed to test thefour hypotheses. A two-way ANOVA was used to test that the mean scores for user satisfaction (the
dependent variable) varied by two factors (independent variables). For example, to test hypothesis
H1, an ANOVA was run with user satisfaction with Web portals as the dependent variable, and gender
and portal type as the independent variables. This was to test if the mean on user satisfaction
with Web portals was equal for each category in the combinations of the two independent variables.
Using the same test, we also tested the controlling effect of each independent variable in the pair
of two (H1a and H1b). For example, in H1a, we tested that after controlling for the factor of
portal type used, the means for user satisfaction varied by gender, or male and female users are
satisfied to different extent.
Table 11-2 shows
the F values and significance level for each hypothesis.
Hypotheses | Dependent Variable | F test of overall model | Independent Variables | F test of each variable term |
---|---|---|---|---|
H1 | Overall Satisfaction | 3.10** | Gender | 1.32 |
Portal | 4.57** | |||
H2 | Overall Satisfaction | 2.59** | Age | 1.75 |
Portal | 4.27** | |||
H3 | Overall Satisfaction | 3.32*** | Experience | 3.09** |
Portal | 5.28** | |||
H4 | Overall Satisfaction | 2.79** | Use | 2.23* |
Portal | 4.58** | |||
Note: In all tables, * p<0.1, ** p<0.05, ***p<0.01 |
We also used least squares means comparisons to identify which groups were significantly
different from each other if a significant F value for the entire model was obtained. We used the
T-test for significance of the tests, which allows for all possible combinations of group means to
be tested. For example,
Table 11-3a lists
the least squares means of each category of work experience, which is after controlling for the
factor of portal type used. And it lists the t-value and significance of each comparison of means
between all different groups. The least squares means and comparison T-tests are shown in
Table 11-3a,
11-3b,
11-4a,
11-4b,
11-5 and
11-6.
Years of Work Experience | LSMean | Less than 1 year | 1-3 years | 4-6 years | 7-10 years |
---|---|---|---|---|---|
Less than 1 year | 3.94 | ||||
1-3 years | 4.00 | 0.60 | |||
4-6 years | 4.04 | 0.77 | 0.33 | ||
7-10 years | 4.22 | 1.14 | 0.90 | 0.68 | |
More than 10 years | 3.24 | -2.96*** | -3.22*** | -3.19*** | -3.01*** |
Portal Type | User Satisfaction (LSMean) | T value |
---|---|---|
Customer Community Portals | 3.99 | 2.32** |
Mega/Horizontal Portals | 3.79 |
No. of hours of Internet Use per week | LSMean | 30 - 60 mi | 1-2 hr | 3-5 hr | 6-10 hr |
---|---|---|---|---|---|
30 to 60 mi | 3.39 | ||||
1-2 hr | 3.84 | 1.74* | |||
3-5 hr | 3.95 | 2.27** | 0.73 | ||
6-10 hr | 4.08 | 2.80*** | 1.61 | 1.12 | |
more than 10 hr | 3.95 | 2.26** | 0.71 | -0.04 | -1.10 |
Portal Type | User Satisfaction (LSMean) | T value |
---|---|---|
Customer Community Portals | 3.94 | 2.14** |
Mega/Horizontal Portals | 3.75 |
Portal Type | User Satisfaction (LSMean) | T value |
---|---|---|
Customer Community Portals | 4.05 | 2.14** |
Mega/Horizontal Portals | 3.85 |
Portal Type | User Satisfaction (LSMean) | T value |
---|---|---|
Customer Community Portals | 3.78 | 2.07** |
Mega/Horizontal Portals | 3.59 |