Study methods were designed by investigators in consultation with the Aboriginal Drug and Alcohol Council of South Australia; the Aboriginal Drug and Alcohol Network, representing Aboriginal alcohol and other drug workers in NSW; and the Aboriginal Health Council of South Australia (AHCSA), the peak body for ACCHSs in South Australia (SA). Ethical approval was obtained from ACHSA and from the Metro South Health Human Research Ethics Committee in Queensland.
Recruitment
To assess the validity of the Finnish method as delivered by the App with a range of drinkers, stratified sampling was used. We aimed to recruit: 20 non-drinkers, 40 non-dependent drinkers and 40 dependent drinkers in each of two states by word-of-mouth in each service. Most of the analyses involved in validating and shortening the scale used in the pilot study require little statistical power. For instance, for the reliability analysis, in order to have sufficient power (80%) to identify a correlation of 0.4 where r = 0.8, and ɑ = .05 a sample size of 46 is required (calculated using the ‘pwr’ package in R). Greater sizes were sought to try to allow analysis of differences between urban and remote sites in sampling, and because of anticipated challenges in ensuring complete data collection. In urban Queensland (Qld), recruitment was based in an Indigenous primary health care service and surrounding community. In South Australia (SA), recruitment centred on a regional ACCHS and a remote Aboriginal community-controlled drug and alcohol day centre (a drop-in service). Individuals were eligible for inclusion if they self-identified as being Aboriginal or Torres Strait Islander and were 16+ years. Exclusion criteria included obvious intoxication. Participants were reimbursed for their time and travel expenses with a store voucher.
Each participant also took part in a semi-structured clinical interview, typically within 2–7 days of completing the App. We set out to have half the participants complete the App before the interview, and half afterwards. To assess the test–retest reliability of App responses, participants were asked to complete the App twice within 2–7 days.
Data collection and instruments
Grog Survey App
The development of the App and selection of its survey items have been described elsewhere [21]. Broadly, the App presented questions on demographics, alcohol consumption (10 items), alcohol dependence (3 items based on ICD-11 [28]), harms to self or others, treatment access and participants’ feedback on using the App.
The App ‘reads out’ the questions in English or in Pitjantjatjara (a language spoken in a region of NT, SA and WA). The App was designed to take no longer than 20 min to complete. Aboriginal field research assistants handed the tablet computer to participants, with brief guidance, then stood to one side, in case there were any challenges. Individuals with no prior computer contact were able to use the App without assistance [29]. The App is designed to work ‘offline’ (without access to the internet) and data are synchronised at the end of each working day to a secure encrypted server at the University of Sydney.
Using an adaptation of the Finnish method of assessing drinking, the App asks respondents to show the date of their four most recent drinking occasions within the last twelve months. Participants are then asked how much alcohol they consumed on each of these occasions. Participants select pictures of the type of alcohol, the container they drank it out of, and how full the container was with alcohol [21]. The App also allowed the participant to describe the alcohol consumption of their drinking group, if that was easier for them, and to then show their share.
Participants were asked: “Some people’s hands shake when they stop drinking or before their first drink of the day. How often does this happen to you?” Responses were indicated on a five-point Likert scale ranging from ‘never’ to ‘most days or every day’.
Data collection for the survey App was facilitated by five Aboriginal research assistants (1 male and 2 female, SA; 2 female, Qld) who were either Aboriginal health workers [3] or Aboriginal health professionals working in drug and alcohol [2]. One day of face-to-face training was provided on how to use the App and on study methods. These research assistants introduced each participant to the App then sat a short distance away to ensure privacy, ready to respond if questions arose.
Clinical interview
The clinical interview was conducted by two female Aboriginal health professionals (one in each state), each with knowledge of local culture, context and language. In keeping with past research with Aboriginal respondents [30], this was considered the most suitable reference standard. It was not considered appropriate to have a non-Indigenous addiction psychiatrist or psychologist, as the gulf in cultural understanding and trust can interfere with assessment quality, particularly in remote regions, but even in urban settings [25]. A semi-structured framework was used by the health professional to record notes on their findings (available from the authors on request). This framework was derived from one used by a respected Aboriginal alcohol and drug worker in regional NSW. It was adapted with the help of the two Aboriginal health professionals, in order to better fit the needs of the recruitment sites. The two Aboriginal health professionals agreed on the goal of a conversational clinical interview of 30 min or less.
Consistent with local clinical practice, the interviewer assessment of drinking focused on the past 14 days. Notes on drinking quantity were recorded on a 2-week calendar showing the days of the week (from Monday to Sunday in each row) so that the client could look along with the interviewer. The approach used was similar to the Timeline Followback [9]. If no drinking, or atypical drinking had taken place in the previous 1–2 weeks, the interviewer asked about an additional 2 weeks (i.e. 3–4 weeks in total).
On each drinking day, the interviewer recorded ‘longhand’ what the participant drank, for example, “two longnecks of West End”, which is the local term for 2 × 750 mL bottles of a brand of full-strength beer. To maximise participant engagement and to keep the interview short, interviewers did not convert responses to Australian standard drinks (10 g ethanol). This was later done by a research assistant (TW).
Interviewers also assessed dependence, by asking current drinkers if they experienced tremors when they stop drinking or cut down. Clarification on the interviewers’ clinical notes was sought by one author (KC) if needed. Data entry was conducted by a research assistant.
Analysis
Standard drinks were calculated by the App itself. All other analyses were performed in the R statistical programming language [31]. As there was a lack of a clear gold standard in use identified for Indigenous samples, validity was determined by triangulating the Finnish method with multiple outcomes. The primary outcomes used to compare the Grog App and clinical interview, was the classification of drinking risk, as defined by current Australian guidelines [32]. We also looked at the Spearman correlations of estimates of consumption from each measure with each other, and to the frequency of withdrawal tremors. Steiger’s z-test [33] was used to test for significant differences between the correlations of consumption data and the presence of withdrawal tremors, between the App and clinical interview.
The mean number of standard drinks that participants consumed on drinking occasions was calculated by taking the average of the number of standard drinks consumed on the most recent four drinking occasions. The last four occasions (Finnish) method of assessing consumption does not use a fixed reference period. Non-drinking days are recorded between the interview and most recent occasion. However, necessarily, the reference period ends with the fourth most recent drinking occasion. On average, this results in over-estimates of drinking frequency. To reduce this bias, for each individual, we extended the reference time period by adding half of the average gap between their drinking occasions to their reference period. To calculate the frequency of drinking occasions, the total number of recorded occasions was then divided by the total reference period (in days). To calculate average daily consumption, we multiplied the average quantity each individual consumed per occasion, by their frequency of drinking occasions.
Following data cleaning, estimates of alcohol consumption by the last four occasions method were compared to estimates from the clinical interview. We examined the correlation between the two estimates of both drinking intensity (drinks per drinking occasion) and average daily consumption for each person.
Sensitivity and specificity were calculated to compare the extent to which the App agreed with the clinical interview when classifying drinkers’ risk (as defined by the National Health and Medical Research Council [32]). The clinical interview was used as the reference standard. To assess convergent validity, we compared alcohol consumption estimated by the App, against the presence of withdrawal tremors (recorded on the App or in the interview). We also compared consumption measured by clinical interview against withdrawal tremors. Finally, test–retest reliability was assessed by correlating consumption across the two occasions when participants used the App.
Classification of drinking risk
Short-term risk from drinking was defined as consumption of more than four drinks on any occasion [32]. Long-term risk from drinking was defined as average consumption of more than two drinks per day [32]. Both these criteria are in keeping with Australian guidelines to reduce the risk from drinking [32]. In addition, a short-term high risk threshold of more than 10 drinks per day was examined based on the higher odds of motor vehicle accidents at 11+ drinks per day in a meta-analysis [34]. Long-term high risk was defined by average daily consumption of more than five standard drinks was examined. This cutoff was based on a meta-analysis showing increased relative risk of cancers at that level [35].
Reference period matching
Comparisons of App and interview consumption initially considered all recorded drinking data. However, because the clinical interview focused on the last 7–29 days while the App collected data on four occasions which could be spread across the year, comparisons were then repeated using only those days which were examined by both the App and interview. To account for differences in participants’ recall of when these occasions occurred, a buffer of 3 days was included. For example, if a date logged in the App occurred within 3 days of a date in the corresponding clinical interview (whether consumption was zero or higher), it was included in the analyses. The median number of matched days was 23.5. All participants had one or more matched days.