The research sample used in this study included 340 students at a large mid-Atlantic university in the USA. The sample size is comparable to similar studies on user satisfaction such as McHaney and Cronan (1998) and McHaney et al. (2002). Although the sample for this study was collected at a higher education institution, it represents the end user population well, because some students were full-time students while others were part-time from a variety of industries and management levels. Table 11-1 provides sample descriptive statistics. As the table indicates, the subjects are distributed evenly in male (52%) and female (48%). As for work experience, about 75% of the subjects have a few years of experience (35% + 41%), while 25% had more than three years of experience. Most of the subjects use computers at home (86%) and school (77%), and close to one half (41%) of the subjects use computers at work. Some subjects use computers at multiple places.
|
Gender |
Male |
Female |
Total* |
|||
|
174 (52%) |
158 (48%) |
332 |
||||
|
Work Experience |
< 1 year |
1 - 3 years |
4 - 6 years |
7 - 10 years |
> 10 years | |
|
115 (35%) |
134 (41%) |
52 (16%) |
14 (4%) |
14 (4%) |
329 |
|
|
Location (multiple choices) |
Home |
Work |
School | |||
|
291 (86%) |
138 (41%) |
261 (77%) |
690 |
|||
|
* N = 340. There are some missing data. |
||||||
The data for this study were collected in 2002 as part of a larger research study on user satisfaction with Web-based information systems. A survey instrument was pilot tested with 30 graduate students and the instrument proved to be valid and reliable. The research questionnaire was then distributed in class to 340 graduate and undergraduate students. While collecting the data, students were informed that participation in this study was voluntary and anonymous. No personal identity information was collected in the survey. Hard copy questionnaires were distributed in class by the researcher. It took about 10 minutes to complete the survey.
Analysis of variance (ANOVA) was performed to test the four hypotheses. A two-way ANOVA was used to test that the mean scores for user satisfaction (the dependent variable) varied by two factors (independent variables). For example, to test hypothesis H1, an ANOVA was run with user satisfaction with Web portals as the dependent variable, and gender and portal type as the independent variables. This was to test if the mean on user satisfaction with Web portals was equal for each category in the combinations of the two independent variables. Using the same test, we also tested the controlling effect of each independent variable in the pair of two (H1a and H1b). For example, in H1a, we tested that after controlling for the factor of portal type used, the means for user satisfaction varied by gender, or male and female users are satisfied to different extent. Table 11-2 shows the F values and significance level for each hypothesis.
|
Hypotheses |
Dependent Variable |
F test of overall model |
Independent Variables |
F test of each variable term |
|---|---|---|---|---|
|
H1 |
Overall Satisfaction |
3.10** |
Gender |
1.32 |
|
Portal |
4.57** |
|||
|
H2 |
Overall Satisfaction |
2.59** |
Age |
1.75 |
|
Portal |
4.27** |
|||
|
H3 |
Overall Satisfaction |
3.32*** |
Experience |
3.09** |
|
Portal |
5.28** |
|||
|
H4 |
Overall Satisfaction |
2.79** |
Use |
2.23* |
|
Portal |
4.58** |
|||
|
Note: In all tables, * p<0.1, ** p<0.05, ***p<0.01 |
||||
We also used least squares means comparisons to identify which groups were significantly different from each other if a significant F value for the entire model was obtained. We used the T-test for significance of the tests, which allows for all possible combinations of group means to be tested. For example, Table 11-3a lists the least squares means of each category of work experience, which is after controlling for the factor of portal type used. And it lists the t-value and significance of each comparison of means between all different groups. The least squares means and comparison T-tests are shown in Table 11-3a, 11-3b, 11-4a, 11-4b, 11-5 and 11-6.
|
Years of Work Experience |
LSMean |
Less than 1 year |
1-3 years |
4-6 years |
7-10 years |
|---|---|---|---|---|---|
|
Less than 1 year |
3.94 | ||||
|
1-3 years |
4.00 |
0.60 | |||
|
4-6 years |
4.04 |
0.77 |
0.33 | ||
|
7-10 years |
4.22 |
1.14 |
0.90 |
0.68 | |
|
More than 10 years |
3.24 |
-2.96*** |
-3.22*** |
-3.19*** |
-3.01*** |
|
Portal Type |
User Satisfaction (LSMean) |
T value |
|---|---|---|
|
Customer Community Portals |
3.99 |
2.32** |
|
Mega/Horizontal Portals |
3.79 |
|
No. of hours of Internet Use per week |
LSMean |
30 - 60 mi |
1-2 hr |
3-5 hr |
6-10 hr |
|---|---|---|---|---|---|
|
30 to 60 mi |
3.39 | ||||
|
1-2 hr |
3.84 |
1.74* | |||
|
3-5 hr |
3.95 |
2.27** |
0.73 | ||
|
6-10 hr |
4.08 |
2.80*** |
1.61 |
1.12 | |
|
more than 10 hr |
3.95 |
2.26** |
0.71 |
-0.04 |
-1.10 |
|
Portal Type |
User Satisfaction (LSMean) |
T value |
|---|---|---|
|
Customer Community Portals |
3.94 |
2.14** |
|
Mega/Horizontal Portals |
3.75 |
|
Portal Type |
User Satisfaction (LSMean) |
T value |
|---|---|---|
|
Customer Community Portals |
4.05 |
2.14** |
|
Mega/Horizontal Portals |
3.85 |
|
Portal Type |
User Satisfaction (LSMean) |
T value |
|---|---|---|
|
Customer Community Portals |
3.78 |
2.07** |
|
Mega/Horizontal Portals |
3.59 |