Development and validation of the Stress Sensation Inventory ( SSI ) to measure stress among students aged 15 to 19 years in Sri Lanka

In-depth interviews and focus group discussions were conducted among adolescents, in order to explore physical symptoms and thoughts experienced and their behavioural responses, when faced with challenging problems and situations. Following this, a brief, self-administered instrument, the Stress Sensation Inventory (SSI) was developed by a group of experts. The SSI was then validated among 127 school-going adolescents, and the scores were compared with the GHQ30 score and the clinical assessments carried out by a psychiatrist. The cut-off point was determined using the receiver operator characteristic (ROC) curve.


Introduction
Stress is an invariable factor in life.However, negative stress may cause responses ranging from tiredness to severe depression.If a situation is unpredictable and uncontrollable, the result is likely to be a stress response.Adolescents who are going through puberty, must meet the changing expectations of others and cope with new feelings (1).The way these challenges are negotiated has critical implications for long-term health and development (2).
Stress in adolescents could present in a myriad of ways, including psychosomatic and psychological features such as dizziness, feeling low, headache and stomach pains, fatigue, lack of appetite, and low level of school satisfaction (3)(4)(5).Although this is a common problem, there is no satisfactory method to measure stress among Sri Lankan adolescents.There are several instruments available internationally to measure stress among adolescents.The Perceived Stress Scale (PSS) is one such instrument which measures psychological stress related to specific situations (6).The PSS shows good reliability (Cronbach's alpha 0.67) and satisfactory concurrent validity when compared with the Hospital Anxiety and Depression Scale-Anxiety (HADS-A) scale and Stress Likert Scale (r= 0.65 and r=0.54 respectively) but it has not been validated for Sri Lanka (7,8).The General Health Questionnaire-30 (GHQ-30) has been constructed for the assessment of the presence of any mental disorder, but since it identifies anxiety, minor depressive illness and transient situational disturbances (e.g., such as feelings of incompetence in coping) it has been used in Sri Lanka to measure psychological aspects of stress (9)(10)(11).Thus none of these scales comprehensively evaluate the effects of stress.Although there are other international scales that measure stress, adaptation is a challenge due to logistic unfeasibility.Therefore there is a need for a method of measuring stress among Sri Lankan adolescents.Thus, the aim of this study was to develop and validate a scale (the Stress Sensation Inventory) to measure stress among students aged 15 to 19 years in Sri Lanka.

Designing of the SSI
In designing the Stress Sensation Inventory (SSI), several scales, namely the PSS, measures used for the World Health Organization's cross national survey ''Health Behaviour in School-Aged Children'' (HBSC) and the GHQ-30 validated for Sri Lanka were studied and relevant items were modified and adapted (6,11,12).Twenty seven in-depth interviews and four focus group discussions (FGDs) were conducted among 15 to 19 yearold school going adolescents.The interviews and discussions explored physical symptoms and thoughts experienced by these students, and their behavioural responses, when faced with challenging problems and situations.In order to maintain homogeneity among members of the focus groups, male and female students in the Ordinary Level and Advanced Level classes were categorised into the four groups.
Recurrent themes that emerged from the interviews and discussions included physical symptoms and thoughts commonly experienced during stressful events, and the ways in which these adolescents behaved at times of stress.Based on these themes, individual and informal group discussions were conducted with experts to identify symptoms related to stress.After preliminary discussions, the principal investigator drafted a list of symptoms.Clarity, appropriateness and suitability of the items were judged by a group of experts including psychiatrists, psychologists, community physicians, educationalists and sociologists.The final items were categorised into different domains and sub-domains.

Development of the composite score for SSI
For each item an un-weighted minimum value of 0 was given to the response ''never'' and a maximum value 3 to the response ''always''.Responses to positively stated items were reverse scored.The total score was calculated by adding individual scores of each item.A higher value indicated an increased likelihood of having experienced stress, while lower values indicated a lesser likelihood of having stress.Based on this, arbitrary cut-off values were calculated, with the highest value range indicating severe stress and the lowest value range indicating presence of minimum or no stress.Thus depending on the level of stress experienced the SSI score was divided into three categories, namely severe (total score 64-96), moderate (total score 32-63) and no stress (total score 00-31).

Validation of the SSI Study sample and methodology
The SSI was pretested on 35 school going adolescents in the Gampaha district, in order to determine acceptance, feasibility, understanding and interpretation of questions, and assess logistical problems.Based on the results, a few changes were made in the wording and sequence of items in the scale.Thereafter the validation of the SSI was conducted in a school with Advanced Level classes, situated in the Gampaha District of Sri Lanka.Female and male students in the age group 15-19 years were included in the study.Non-Sinhala speaking students were excluded and adolescents with learning disability, hearing or visual impairment, and history of stroke, head trauma, psychotic illness or depression were also excluded.
The sample size was calculated using the desirable lowest sensitivity and specificity.As this was a screening instrument, sensitivity was required to be high and specificity had to be acceptable.Hence, 80% sensitivity and 80% specificity were used to calculate the sample size (13).Accordingly the minimum sample size required was 124.Convenient sampling was used and all the students who were in the age group of 15 to 19 years in the selected school were recruited for the study.

Reliability of the instrument
Internal consistency of the responses to questions coming under each sub-domain was measured by calculating Cronbach's alpha using the statistical software package SPSS (14).

Face validity of the SSI
The face validity was ascertained during the discussions with experts and during the stage of pretesting.

Content validity
The content validity was established by obtaining information from in-depth interviews and FGDs with adolescents, discussions with experts, review of other scales, and obtaining the agreement of the expert panel using the Delphi Method (15).

Construct validity
Stress is associated with mental disorders as well as suicide (16,17).The GHQ-30 measures the probability of having a mental disorder and has been validated for use in Sinhalese in Sri Lanka to measure stress (10,11).Thus, it was decided to use the GHQ-30 to assess the construct validity of SSI.The cutoff value for the GHQ-30 was taken as 5/6 (10), where a score of 6 indicated psychological distress.The Pearson correlation coefficient was used to measure the correlation between the scores of the two instruments.Kappa agreement was used to detect the agreement between the two instruments.

Criterion validity
Criterion validity was assessed by comparing the SSI results with the gold-standard for identifying stress.Since there is no absolute 'gold standard' for assessment of stress, the clinical judgment of a consultant psychiatrist was considered as the 'gold standard'.All participants were invited to complete the SSI.Subsequently, on the same day, participants were interviewed by a psychiatrist who was blind to the results of the SSI.To ensure uniformity of the interviews, the psychiatrist developed and used a structured guide.The Development and validation of the Stress Sensation Inventory criteria used for diagnosing stress by the psychiatrist included physical symptoms, cognitive symptoms e.g.concentration difficulties, emotional symptoms, selfperceptions and coping.Based on the psychiatrist interviews, the participants were categorized into three categories -namely those suffering from severe, moderate and no stress.
The cut-off point for the SSI was determined using the receiver operator characteristic (ROC) curve.Various cutoff points were selected and sensitivity and specificity at each point was determined.Then sensitivity as a function of specificity was graphed.
Ethical clearance was obtained from Ethical Review Committee of the Faculty of Medical Sciences, University of Sri Jayewardenepura and informed written consent was taken from all parents/legal guardians of the participants.

Results
A total of 127 adolescents were included, of whom 56% were males.The mean age was 16.1 years (SD=1.0).Two third were 15 to 16 year olds.The group comprised mainly of Sinhalese (94%) and Buddhists (91%) and the majority belonged to social class II.

Reliability
Cronbach's alpha values for the different domains of the SSI are given in Table 1.

Construct validity: Comparison with GHQ-30 questionnaire
Table 2 shows the results of the comparison of the GHQ-30 and the SSI.The Kappa agreement was 0.35 and Pearson correlation coefficient was 0.38 (P < 0.001).

Criterion validity
The level of stress based on the SSI score and judged by the psychiatrist interview was compared to determine criterion validity (Table 3).The criterion validity of the SSI was further assessed by calculating the sensitivity, specificity, positive and negative predictive values and false positive and false negative rates (13).
The sensitivity and specificity were calculated using different cut off points of scores of the SSI.The optimal cut off point was obtained using the receiver operating characteristic (ROC) curve (Figure 1).At a cut-off point of 31.5, the sensitivity was 84.1% and specificity was 72.3%, which provided the best discrimination between students who were suffering from stress, and those who were not affected by stress.

Figure 1. Receiver Operating Characteristic curve of the SSI
The positive likelihood ratio was 3.0 and the negative likelihood ratio was 0.22.The SSI had a Positive Predictive Value (PPV) of 61.7%, whereas the Negative Predictive Value (NPV) was 90.0%.The false positive rate was 27.7% and the false negative rate was 15.9%.

Discussion
Stress can be defined as a condition occurring when there is an ''imbalance between demands and resources'' or when ''pressure exceeds one's perceived ability to cope'' and (18).The stress response has different domains.The SSI clearly identified these different domains namely 'psychosomatic' and 'psychological', which is an advantage when compared to tools which measure stress in only one domain.This is particularly important since the presentation of stress among students in Sri Lanka is not equally distributed among these domains, as was identified during FGDs and indepth interviews.Thus the SSI was developed to cover all the domains of the stress response.
The reliability of the SSI, as assessed by the Cronbach's alpha, was 0.8 for the total score and was more than 0.6 for all the sub-domain scores.A Cronbach's alpha value of above 0.6 is considered satisfactory and above 0.7 is considered excellent (19)(20)(21)(22).
Face validity of the SSI was ascertained during pretesting.A comprehensive literature review, formal and informal discussions with experts, in-depth interviews and FGDs with students were carried out to obtain a high content validity.Assessment of construct validity revealed a Kappa agreement of 0.35 between the GHQ-30 and the SSI scores.According to Altman this indicates a fair level of agreement between the SSI and GHQ-30 (23).Although the strength of the agreement was not strong, the Pearson correlation coefficient indicated a significant correlation between SSI and GHQ-30 (p<0.001).The GHQ-30 has been primarily constructed to assess the probability of having amental disorder and is not comprehensive for stress assessment (9)(10)(11).This may be the reason why the Kappa agreement was not highly satisfactory.However, the correlation between the two tools was significant and the reason for this could be that the two instruments measured similar psychological features to some extent.
Criterion validity in the context of SSI is the ability of the instrument to correctly distinguish those who were stressed from those who were not.This was assessed by comparing the SSI results with the ''gold-standard'' for identifying stress, which was considered to be the clinical judgement of a consultant psychiatrist.Ideally, a screening test should have a high sensitivity and a high specificity; however there is a trade-off between the sensitivity and specificity.A balance of specificity and sensitivity is placed at the cut-off point between the decision of arbitrary normal and abnormal values (24).Using the receiver operator characteristic (ROC) curve, a cut-off point of 31.5 was chosen for the SSI, which gave a sensitivity of 84.1% and specificity of 72.3%.These results are acceptable for a screening tool.The PPV and NPV of SSI were 62% and 90% respectively.The PPV and NPV are the proportions of positive and negative results in the tool which are true positive and true negatives (25).The PPV value for SSI is low.The predictive values depend on the prevalence and specificity and to a lesser extent, on the sensitivity.Even with high sensitivity and specificity, if the prevalence is low, the PPV may be low (26).On the contrary, likelihood ratios (LR) are not affected by the prevalence.The LR of a result is the probability of that result being present in patients with a condition divided by the probability of the same result being present in patients without the condition (27).The likelihood ratio can be used to assess how good a diagnostic test is and to help in selecting appropriate diagnostic tests (27).Further, according to this study results, the false positive rate of the SSI was 28% and the false negative rate was 16%.The positive likelihood ratio was 3.0 and negative likelihood ratio was 0.22.Having a positive likelihood ratio of 3.0 means that a student identified as having stress via SSI is three times more likely to be from the group of students who truly have stress than from the group of students who does not have stress.
Limitations of this study include the fact that the study was carried out in an urban setting in the Gampaha district of Sri Lanka.The socio cultural factors operating in more

Table 1 .
Internal consistency of different domains of the Stress Sensation Inventory (SSI)

Table 2 .
Comparison of mental wellbeing and stress as identified by the GHQ 30 and SSI

Table 3 .
Comparison of level of stress as identified by the SSI and psychiatrist interview