Validation of the 4AT tool for delirium assessment in specialist palliative care settings: protocol of a prospective diagnostic test accuracy study [version 1; peer review: 2 approved]

Background Delirium is a serious and distressing neuropsychiatric condition, which is prevalent across all palliative care settings. Hypoactive delirium is particularly common, but difficult to recognize, partly due to overlapping symptoms with depression and dementia. Delirium screening tools can lead to earlier identification and hence better management of patients. The 4AT (4 ‘A’s Test) is a brief tool for delirium detection, designed for use in clinical practice. It has been validated in 17 studies in over 3,700 patients. The test is currently used in specialist palliative care units, but has not been validated in this setting. The aim of the study is to determine the diagnostic accuracy of the 4AT for delirium detection against a reference standard, in hospice inpatients. Methods 240 participants will be recruited from the inpatient units of two hospices in Scotland. If a patient lacks capacity to consent, agreement will be sought from a legal proxy. Each participant will complete the 4AT and a reference standard assessment based on the diagnostic delirium criteria in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). This will be supplemented by tests of cognition and attention, including reverse days of the week, counting down from 20 to 1, Vigilance 'A', the Observational Scale for Level of Arousal, the modified Richmond Agitation Sedation Scale and the Delirium Rating Scale-Revised-98. The assessments will be conducted in a randomized order by two independent clinicians, who will be blinded to the results until both are complete. Primary outcomes will be the sensitivity and specificity of the 4AT in detecting delirium. Discussion The findings will inform clinical practice regarding delirium assessment in palliative care settings. Trial registration ISRCTN ISRCTN97417474 (21/02/2020).


Plain language summary
Delirium is a serious and distressing condition, which commonly affects people with terminal illness. Approximately a third of patients will have delirium on admission to hospices. Delirium usually develops over hours to days. It can cause confusion, and problems with memory, concentration, perception (may develop hallucinations) and altered consciousness (people can become agitated or drowsy and withdrawn). It is important to diagnose delirium early, as it increases the person's chance of recovery. When a person has only hours or days to live, their delirium is less likely reversible, but early detection is still beneficial, as it helps healthcare professionals decide how they can best support the person and their family.
The 4'A's test or 4AT is a short, easy to administer bedside test, used by healthcare professionals to determine if patients may have delirium. Research has shown the 4AT to be an effective test in hospital patients. It is currently used in hospices, but research is needed to review its use in this setting. This article describes the proposed study to examine the effectiveness of the 4AT in detecting delirium in terminally ill hospice patients.
Patients admitted to two hospices in Scotland will be invited to take part in the study. Those participating in the study will undergo two assessments -the 4AT and another more detailed delirium assessment.
This study will provide evidence about the accuracy of the 4AT in detecting delirium in terminally ill hospice patients. If the study shows the 4AT to be effective, it may be more readily used in routine practice, than other more complex and lengthy delirium tests. If delirium is detected earlier, it may lead to better care for these patients and their families.

Background
Delirium is a serious and distressing neuropsychiatric condition, characterized by an acute disturbance in attention, awareness and cognition. Delirium severity may fluctuate throughout the day, and be associated with disturbances in the sleep-wake cycle.
Cognitive changes may impact memory, orientation, language, visuospatial ability and perception 1 . Delirium can be highly distressing for patients and their families and is associated with poor outcomes 2-6 .
Delirium is extremely common across all palliative care settings 7,8 . A recent systematic review estimated the median (range) point prevalence of delirium on admission to palliative care inpatient settings as 32% (6.6%-73%) and period prevalence prior to death as 75% (58%-88%) 7 . Hypoactive delirium is more common than other subtypes in palliative care 9-11 but may be less noticeable and go under-recognised due to overlapping symptoms with depression, dementia and fatigue 11-13 .
International guidance recommends routine assessment for risk factors of delirium on admission to hospital and other care settings, and subsequently if there are any fluctuations or changes in behaviour or cognition 14-18 . Delirium assessment tools are recommended and may lead to earlier detection. Improved detection may lead to better management, including investigation and treatment of the underlying cause and/or better symptom control, resulting in more favourable outcomes for both patients and their families. • as research assessments: The Delirium Rating Scale (DRS) and the Delirium Rating Scale Revised-98 (DRS-R-98) 26 . Table 1 shows examples of validation studies in palliative care and oncology populations. The SQiD is a quick and easy to use tool, which can be incorporated into the admission process.
If the SQiD scores positive, this could trigger a more detailed assessment for delirium. The CAM performed poorly in this study, likely because the assessors had limited training in its use.

' A's test (4AT) 22
None n/a n/a Delirium Rating Scale-revised-98 (DRS-R-98) 26 and Delirium Rating Scale (DRS) Used as reference standard delirium assessments in research studies 11,30,32,37,38 The DRS-R-98, a clinician-rated 16-item scale with 3 diagnostic and 13 severity scales, is able to distinguish reliably from depression and dementia 26 . Use in clinical practice is limited because of training required prior to use and time taken to administer (at least 20-30 minutes with the patient, plus additional time to gather informant history and review case notes) 11,27 .
The Memorial Delirium Assessment Scale (MDAS), a 10-item scale, used for delirium detection and severity monitoring, takes approximately 10 minutes with the patient, plus additional time to gather informant history 30 . It has a simpler format and is easier to rate than the DRS-R-98, but still requires moderate training to administer 27 . A study in a palliative care inpatient unit demonstrated 97% sensitivity and 95% specificity (cutoff score = 7) 31 .
The CAM and its shorter variants are quicker to administer than either the MDAS or DRS-R-98 (approximately 5-10 minutes), but still require moderate training prior to use, as well as additional cognitive testing 27 . Aspects requiring subjective judgement by the assessor can be more complex and time-consuming to rate, plus there is limited advice for scoring non-verbalizing patients 39 . This may account for high levels of sensitivity and specificity in some studies 33,34 , but lower results when scored by those less experienced 23,33 .
The informant tool, the Single Question in Delirium (SQiD), which asks 'Do you feel that (patient's name) has been more confused lately?', has been validated in oncology patients (80% sensitivity and 71% specificity), but not in palliative care patients 23 . Use of the SQiD is limited when an informant is unavailable. The 4AT is recommended for identifying patients with delirium 14 , and is currently used in palliative care settings 41-43 , but has not been validated for delirium screening in a terminally ill population.

Study objective
The study objective is to determine whether the 4AT is a valid tool for delirium detection in specialist palliative care settings. The diagnostic accuracy of the 4AT will be compared against a reference standard delirium assessment, based on the diagnostic criteria of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) (

Study overview
In developing this protocol, we drew on the study design and procedures of a previous 4AT validation study conducted in emergency departments and acute medical wards 39 . This study is a prospective diagnostic accuracy study of the 4AT in a representative sample of hospice inpatients. Each participant will independently complete the 4AT and the reference standard delirium assessment based on DSM-5 criteria. Study procedures are described in Figure 1.

Setting
The study will take place at two hospices in Scotland. The hospices have 17 and 24 inpatient beds, respectively, and admitted a combined total of 700 patients in 2019. Patients with advanced progressive or incurable disease are admitted for complex end of life care, or because they have uncontrolled pain and/or other complex physical or psychological issues, that cannot be managed in other care settings.

Participants and sample size
Participants admitted acutely to the hospices will be included if they are aged 18 years or over. Patients will be excluded if they are comatosed, unable to communicate in English, are severely dysphasic or have a combined severe hearing and visual impairment, which would limit participation in the study's tests. Patients will also be excluded if there is a high level of patient and family distress, as judged by the clinical team, or the patient has an acute life-threatening illness requiring timecritical intervention (e.g. suspected spinal cord compression). We estimate that 80% of those recruited will complete the assessments, allowing data analysis on at least 200, slightly above the 176 needed.
Screening Eligibility screening will take place every morning, and opportunistically, if practical, throughout the working day, as new patients are admitted to the hospice inpatient units. The clinical care team will identify potentially eligible participants by considering a checklist with the inclusion and exclusion criteria, as shown in Table 3. For patients screened as ineligible, age and reason for ineligibility will be recorded. If the reason for not approaching the patient later resolves (e.g. high level of patient and/or family distress or acute life-threatening illness requiring time-critical intervention), the patient may be approached on another occasion.

Inclusion criteria
• Aged 18 years or over • Acutely admitted to the specialist palliative care inpatient units.

Exclusion criteria
• Coma • Unable to communicate in English (The cognitive tests used have not been validated in non-English language speakers, hence the study only includes patients who can communicate fluently in English).
• Severe dysphasia • Combined severe hearing and visual impairment, which would limit participation in the study's tests.
• High level of patient and family distress, as judged by the clinical team.
• Acute life-threatening illness requiring time-critical intervention (e.g. suspected spinal cord compression) Recruitment processes for participants with and without capacity Eligible patients will be approached by the clinical team and asked if they would like to hear more about the study. If agreeable, the clinical researcher will provide verbal and written information about the study and invite them to take part. As part of this discussion, the patient's capacity to consent to participation in the study will be assessed. If the patient is assessed as lacking capacity to decide, their legal proxy will be approached. The legal proxy could be their Welfare Attorney, Guardian or nearest relative (this is the order in which they will be approached). They will be asked to consider the previously expressed views of the patient, and if they think the patient would have wanted to participate in the study. The Adults with Incapacity (Scotland) Act 2000 permits a legal proxy to consent on behalf of an adult with incapacity 45 .
Due to the nature of delirium, it is possible the participant's capacity to consent may fluctuate. If participants with capacity continue with the assessments shortly after giving consent, they are unlikely to lose capacity within this short space of time. However, if there is a longer period between the capacity/consent process and assessments, it is possible that a participant's capacity may be lost or regained in the intervening period. The researcher will need to ensure that the capacity and consent remain valid just prior to the assessments being completed. That is, if a participant later regains their capacity after proxy agreement has been obtained, they will be given an opportunity to provide informed consent for themselves. However, if the researcher becomes aware the previously competent participant has now lost capacity, the assessments will continue in view of their previous consent.
Participants are free to withdraw from the study at any time, or the researcher may withdraw a participant. The reason for their withdrawal will be documented, if available, and data collected up to that point, will be used.

Training of data collectors
The 4AT assessors are hospice inpatient nurses and doctors, who completed delirium training, either during preparation for the study and/or as part of their professional training. Prior to the study, the 4AT was routinely used on admission to the inpatient units, and at other times when delirium was suspected.
The reference standard assessors are also part of the clinical team. Prior to study recruitment, these assessors completed additional training in capacity assessment, obtaining consent and delirium assessment with a Post-Doctoral Research Fellow and psychologist (ZT) and the Chief Investigator (AMJM), to ensure competence. Good Clinical Practice was also completed.

Index test
The 4AT takes a few minutes to complete and is described earlier in this protocol 22 .

Reference standard assessment
The reference standard assessment may take up to 20 minutes to complete and is based on the delirium diagnostic criteria in the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) (  26 . A diagnosis of dementia or learning disability will be recorded. The assessor will review the patient's clinical records and speak with someone who knows the patient well, such as a member of the clinical team and/or relatives (with the patient's consent).
Following the reference standard assessment, the participant will be grouped into one of four categories -delirium present, possible delirium, no delirium, or undetermined, as defined in another recent delirium study 49 (Table 4).
Where there is uncertainty of the patient's categorisation, the reference standard assessor will discuss these 'challenging cases' with an expert panel. This panel includes a Consultant in Palliative Medicine (JS) and the Chief Investigator, who is a

Category Criteria
Delirium present All 5 of the DSM-5 delirium core diagnostic criteria are positive Possible delirium Some DSM-5 delirium diagnostic criteria are positive (i.e. some features of delirium are present), but not all, due to missing information (perhaps about the history of onset of symptoms).
No delirium present The core criteria are negative for delirium.

Undetermined
Some, but not all, DSM-5 delirium criteria are positive. This usually represents a resolving or subsyndromal delirium.
Consultant in Medicine of the Elderly (AMJM). The panel will be blinded to the results of the 4AT, until the final categorisation of the reference standard assessment is complete.

Data collection
Each participant will complete the 4AT and a reference standard delirium assessment by two independent clinicians. The assessor, who conducted the capacity and consenting process, will always administer the reference standard assessment. Given the fluctuating nature of delirium, the two assessments will be completed within a maximum three-hour time period (target interval 15 minutes). During this period, there will be no communication about the participant between the two assessors, until both assessments are complete (other than to arrange the timing of assessments).
The order of the two assessments will be randomised in a 1:1 ratio. The randomisation allocation will occur immediately following recruitment. An administrator at each site, who is independent of the assessments, will use a block randomised list to direct the order of the assessments.
Once both assessments are complete, the outcome of the reference standard assessment will be communicated verbally to the clinical team looking after the patient, in accordance with the consent process. This is because a provisional research diagnosis of delirium may result in improved patient care.
Data recording, storage and monitoring Data will be recorded on paper case report forms and kept securely in locked cabinets, in offices with limited access. Study data collection forms (4AT and reference standard assessment results etc.) will be identified by a participant's unique identification (ID) number only, to maintain confidentiality. Records with the participant's name and other personal identifiers (eligibility and recruitment logs, consent forms etc.) will be stored separately from data collection forms with the participant's ID only.
Clinical researchers will transcribe the data on case report forms into secure databases within Marie Curie Cancer Care IT networks in Edinburgh and Glasgow. The databases will only be accessible to clinical researchers and the Research steering committee, who will be responsible for monitoring the data quality.

Data analysis
The diagnostic test accuracy of the 4AT for delirium detection in specialist palliative care inpatient populations, versus the reference standard, will be determined using sensitivity, specificity, and positive and negative predictive values. The exact binomial 95% confidence interval will be reported for each measure. A ROC curve analysis will be performed, and the area under the ROC curve and its 95% confidence interval will be reported. Analyses will be completed using IBM SPSS Statistics.
Participants with missing data from the 4AT and reference standard assessments will be included in the statistical analysis, if there is enough information to decide on categorisation. Cases with insufficient information will be excluded, and expert panel advice will be sought where there is uncertainty.
Data protection Data will be collected, stored and handled in accordance with guidance from Marie Curie Research Governance committee, the sponsor ACCORD (the Academic and Clinical Central Office for Research and Development) and the NHS Scotland A Research Ethics Committee.
The principle investigator will have direct access to their own site's data, and to the other site's on request. To ensure confidentiality, the Research steering committee will only have access to data that has had identifiable participant information removed.
Personal data will be stored securely for a maximum of three years to allow full analysis of the data. Study documentation will not be destroyed without permission from the sponsor.

Study oversight
There is a process for reporting adverse events, however this is a relatively 'low risk' study -the assessors are trained palliative care nurses and doctors, and the study tests are already used in clinical practice. The sponsor, ACCORD, will be responsible for external oversight, which may involve monitoring and audit of study activity and documentation.

Study status
Recruitment commenced in October 2019. We initially anticipated that recruitment would run for up to 15 months, however recruitment was temporarily stopped in March 2020 due to the coronavirus disease 2019 (COVID-19) pandemic. Recruitment will resume as soon as permitted, in line with research and clinical governance requirements.

Dissemination of information
The anonymised data will be shared on the ISRCTN Registry within 18 months of trial completion, and we hope to publish the results in an open access peer reviewed journal. The trial was registered retrospectively with ISRCTN on 21 st February 2020 (ISRCTN97417474).

Reporting guidelines
This protocol is reported in line with the SPIRIT guidelines 50 . A completed SPIRIT checklist for this protocol is available on the ISRCTN registry (ISRCTN97417474).

Discussion/implications
A recent survey of UK palliative care specialists reported the majority (68%) only screened for delirium when suspected clinically, and few (5%) screened routinely on admission to palliative care units 43 . Only a third of respondents (37%) used assessment tools to screen for delirium, despite international guidance advocating their use. Inadequate delirium training and guidance, as well as the 'complexity of patient's conditions' were perceived as barriers to delirium screening. The consequence of health care professionals not using assessment tools, is that delirium, particularly the hypoactive subtype, might go unrecognised and hence untreated in these patients.
The 4AT is a delirium assessment tool with strong evidence of validity in hospitalised patients. Whilst it is used in specialist palliative care settings, it has not been validated. If this study shows the 4AT to be an effective tool in this population, it may be more readily adopted into routine practice than other more complex and lengthy tools. If delirium is detected earlier, it may lead to better outcomes for these patients and their families. The findings from this study will provide evidence for the use of the 4AT in hospice settings, and help inform clinical guidelines and practice in relation to delirium assessment in palliative care.

Data availability
No data are associated with this article.
may provide useful evidence with respect to the utility/ possible burden of these tools in this patient population.
Similarly as this study is being conducted in hospice inpatients, it would be useful to record the Palliative Performance Status, or similar performance status tool, of the patient at the time of the 4AT and reference standard assessments as performance status may predict a patient's prognosis. I did not see performance status explicitly mentioned in the protocol.
Inclusion and exclusion criteria: Figure 1: 2nd box: Should it be "patient meets the inclusion or exclusion criteria"? ○ Table 3: Check language used for inclusion criteria, as a specialist inpatient palliative care unit is usually in an acute care hospital. The term "inpatient hospice" is used in Abstract and Plain Language Summary, so perhaps hospice inpatient unit is a better term here? ○ Recruitment processes: Given that trained reference standard assessors need to be available, is the study running seven days a week, or just Monday to Friday? The authors state that "the modified Richmond Agitation-Sedation Scale" would be used, so I had been anticipating the "Modified RASS" as developed by Chester et al., 3 but see that the citation is for another modified version of the RASS that was used in older adults in an emergency department.
○ If a patient is found to be "unassessable" for DSM-5 category 'C', will they also be categorised as "possible delirium" ( The MDAS is listed as the 1 st tool in Table 1, but is not listed as a delirium tool in the last paragraph of Page 3. ○ Table 1: There is another validation paper in cancer patients that has recently been ○ published on the SQiD in BMC Cancer 6 .
In data protection paragraph, typo for 'principal' investigator. Suggest add initials here for the PI.