Psychometric analyses in SPSS,

Measurement and Assessment (M&A) II Assignment

1 M&D II / M&A II 2022-2023 Assignment

M&A II / M&D II 2022-2023. Individual assignment. English version

Measurement and Assessment (M&A) II Assignment

This M&A II assignment consists of carrying out psychometric analyses in SPSS, and reporting the

results. This is an individual assignment, so you are required to do the analyses and write the

research report by yourself. The research report should be comprehensible for people who are not

familiar with the details of psychometric analyses.

Below the required analyses are detailed. You can do all these analyses in SPSS. In reporting the

results, you can copy-and-paste SPSS tables, if you like, but tables should include your own table

caption (titles) and table number and you should remove non-relevant information from the tables.

The report should include clear statements on the reason for the analyses (what are the research

questions?) and the meaning of the results (what are the answers to the questions?).

Word limit

The word limit is 1600 words (not including tables).

Writing and plagiarism checking

The report should be written in clear and correct English. Make sure your report contains no spelling

or grammatical errors (please run a spelling check). Assignments with too many spelling or

grammatical errors will not be graded. You are required to do this assignment individually and to

write the report individually. Note: all reports will be submitted to a rigorous plagiarism checker.

Relationship to lectures and literature

This assignment relates directly to

Furr chapters: 4 (exploratory factor analysis)

Workbook chapters: 2 (logistic regression), 3 (factor analysis)

The relevant PowerPoints: Week 2 Lecture 1, Week 2 Lecture 2, Week 3 Lecture 1

Schedule

When To do Who?

Friday 4 Nov 2022 Assignment on Canvas (rubrics will follow

shortly) Dirk Pelt

Sunday 27 Nov 2022 Hand in first version before 23:59 Students

Wednesday 14 december Feedback before 23:59 Tutors

Friday 23 Dec 2022 Hand in final version 23:59 Students

Sunday 22 Jan 2023 Give end grade before 23:59 Tutors

2 M&D II / M&A II 2022-2023 Assignment

M&A II / M&D II 2022-2023. Individual assignment. English version

Assignment: Neuroticism, Agreeableness & problematic smartphone use

In a pilot study on the psychological consequences of problematic smartphone use among young

adults, a short personality questionnaire was administered to a sample of N=259 participants

between 17 and 24 years. The test measures the personality traits Neuroticism and Agreeableness

by means of 10 Neuroticism and 10 Agreeableness items. In addition, a close acquaintance of each

participant was asked to rate their problematic smartphone use (PSU) by responding to the question

“Does he/she spend too much time on their phone?” (coded 0=no, 1=yes). This variable is called PSU.

The aim of the pilot study was to determine whether:

1) the intended latent factors Neuroticism and Agreeableness are found in the data;

2) the psychometric quality of the personality questionnaire is sufficient;

3) Neuroticism and Agreeableness predict the variable PSU.

The data. The sample comprises 259 young males and females between 17 and 24 years. The 10

Neuroticism items are labelled N1 through N10, the 10 Agreeableness items are labelled A1 through

A10. The actual items, i.e., the content, can be found in the variable labels (e.g., item N5 reads “I

rarely get irritated”). The response format is a 5 point scale, with score 1 indicating “disagree

completely” and 5 indicating “agree completely”. A higher score indicates a greater degree of

Neuroticism or Agreeableness. In addition, as mentioned above, there is the dichotomous variable

PSU (0 means “no problematic smartphone use” and 1 means “problematic smartphone use”). The

SPSS system file containing the data is called NeurAgreePSU.sav.

Note. Some items were negatively formulated, meaning that agreeing with an item actually meant a

lower score on the trait. However, these items have already been reverse scored, so when you

calculate sum scores (see below), you can just add up all the item scores.

The analyses (SPSS menu navigation is discussed below). The present aim is threefold.

The first two aims are investigating whether the intended two-factor structure is found and to assess

the psychometric quality of the items:

Dimensionality & Item quality. Carry out a factor analysis on the 20 Neuroticism and Agreeableness

items.

1. Fit the two common factor model to the data using the maximum likelihood method using

oblique rotation. Investigate the factor loadings (i.e., in the Pattern matrix). You will see that

there are two items (one Neuroticism item and one Agreeableness item) that can be considered

problematic. Identify these two items, and repeat the factor analysis without these two items.

Hint. Recall from Workbook CH3 that: “in general the items with a loading <.30 are

considered as items that do not fit well within the set of questions, the questionnaire, or the

factor.” Also, an item can be considered problematic if it loads higher on a factor other than

the one it is supposed to measure.

2. Interpret the scree plot and eigenvalues of the factor analysis based on the 18 remaining items.

Describe both rules of thumb for the number of factors to extract, and explain for each if the

rule of thumb is in line with an extraction of two factors. Either way, we will continue with the

extraction of two factors.

3. Report the factor loadings, the communalities, and the factor correlation.

3 M&D II / M&A II 2022-2023 Assignment

M&A II / M&D II 2022-2023. Individual assignment. English version

The third aim is to analyze the relationship between personality and problematic smartphone use.

Logistic regression analysis. Determine whether Neuroticism and Agreeableness predict problematic

smartphone use (PSU). To this end, first calculate the test scores by summing the 9 Neuroticism

items into a single score and by summing the 9 Agreeableness items into a single score (so in the end

you have two sum scores). Calculate the z-scores of the test scores by standardizing the test scores.

Use the standardized variables as the predictors in the logistic regression analyses. Do the analysis

twice: once with the Neuroticism z-scores as the predictor, and once with the Agreeableness z scores as the predictor. Include in your report the relevant null hypotheses, and the alpha level of

0.05.

Do neuroticism and agreeableness predict the probability of problematic smartphone use (PSU=1)?

If so, evaluate only for the variable(s) that significantly predict PSU the(ir) predictive value. Do this as

follows:

Calculate and report

1) the unconditional probability of PSU=1, and

2) the probability of PSU=1 given the mean of the z-scores (zero) and

3) the probability of PSU=1 given the mean plus one standard deviation of the z-scores. Because the

predictor is standardized, the mean equals zero and the mean plus 1 standard deviation equals

0+1=1. So in your discussion, include:

prob(PSU=1), i.e., the unconditional probability for PSU=1, and the following conditional

probabilities

prob(PSU =1|zscore=0) = 1 / (1 + exp(-(b0 + b1*0)) and

prob(PSU =1|zscore=1) = 1 / (1 + exp(-(b0 + b1*1)),

in other words, the conditional probabilities of PSU =1 given z-score = 0 (the mean) and z-score = 1

(the mean + 1 SD).

Discuss, based on (your subjective evaluation of) these probabilities, the strength of the predictive

value of each of the personality traits. Which of the two personality traits is a stronger predictor of

PSU? Can you come up with an idea for why one trait is a better predictor than the other? For a

refresher on the Big Five personality traits, see https://www.youtube.com/watch?v=IB1FVbo8TSs.

Report sections

Carry out the required analyses in SPSS (instructions are given below), and write a report on the

basis of the results. Your report should include:

1) Title page. Title page with a sensible title, your name and student number, and the word count

(make sure that the word count is < 1600).

2) Introduction. Introduction with research problem and the aims of the psychometric analyses.

3) Sample and variables. Brief description of the variables and the sample (including gender and

age). Provide descriptives (table 1 with means and standard deviations of the items and the

variables PSU, sex and age).

4) Method section. A method section in which the methods are discussed briefly – which analyses

are carried out (factor analysis and logistic regression analyses) and which questions do these

answer? In the case of the logistic regression analyses, mention the null hypotheses and the alpha

level of 0.05 (i.e., α=.05).

4 M&D II / M&A II 2022-2023 Assignment

M&A II / M&D II 2022-2023. Individual assignment. English version

5) The results (2 tables and 1 figure).

5A) Describe results of the factor analyses (table 2 with rotated factor loadings and

communalities based on the 18 items). Include the scree plot (figure 1). Report the

correlations between the common factors in the text and interpret it.

The following questions need to be addresses in your report. First, looking at the content of

the items of the two items that you have removed from the questionnaire after the first

factor analysis, can you come up with a reason for why these items might not function so

well? Second, given that the questionnaire is supposed to measure the two latent variables

Neuroticism and Agreeableness, are the results interpretable as such? That is, is the

intended two-factor structure found in this dataset? Third, are the questionnaire items good

indicators for Neuroticism and Agreeableness?

5B) Describe the results of the logistic regression analyses (table 3 with the parameter

estimates, standard errors and p-values).

6) Conclusions. Conclusions with a clear answer to the question concerning the psychometric

characteristics of the test and the predictive value of the Neuroticism and Agreeableness scores. Is

the questionnaire acceptable in terms of interpretation of the factor structure and is the quality of

questionnaire items adequate? Do the personality traits predict the variable PSU? Rather than only

focusing on statistical significance also focus on predictive value, as expressed in terms of

conditional probabilities relative to unconditional probabilities (see above for explanation on this).

5 M&D II / M&A II 2022-2023 Assignment

M&A II / M&D II 2022-2023. Individual assignment. English version

SPSS menu’s

The SPSS data file is called NeurAgreePSU.sav.

Factor analysis (workbook chapter 3)

Menu’s: Analyze, Dimension reduction, Factor

In the menu under “Extraction” choose the following options. Make sure to choose Maximum

likelihood.

In the menu under “Rotation”, choose “Direct Oblimin”.

6 M&D II / M&A II 2022-2023 Assignment

M&A II / M&D II 2022-2023. Individual assignment. English version

Sum scores and z scores.

To calculate the test scores (i.e., sum scores) based on the items scores, you can execute the

following syntax and fill in the item names in the open … spots (see workbook Chapter 2 Appendix:

Using SPSS syntax). Please note that a final dot (.) is needed after each statement.

COMPUTE Neuroticism = N1 + … + … + … + … + … + … + … + …. .

EXECUTE.

COMPUTE Agreeableness = A1 + … + … + … + … + … + … + … + …. .

EXECUTE.

You can obtain z-scores of the sum scores by choosing “Analyze” and “Descriptives”, and clicking on

the option Save standardized values as variables.

Logistic regression (workbook chapter 2)

Menu’s: Analyze, Regression, Binary logistic