Objectives: Discrete-event simulation is a widely used computational method in health services and health economic studies. This systematic scoping review investigates to what extent authors share computer models, and audits if sharing adheres to best practice. Data sources: the Web of Science, Scopus, PubMed, and ACM Digital Library databases were searched between 1st January 2019 till 31st December 2022.Eligibility criteria for selecting studies: Cost-effectiveness, Health service research and methodology studies in a health context were included.Data extraction and synthesis: the data extraction and best practice audit were performed by two reviewers. We developed best practice audit criteria based on the Turing Way and other published reproducibility guides.Main outcomes and measures: We measured the proportion of literature that shared models; we report analyses by publication type, year of publication, Covid-19 application; and free and open source versus commercial software. Results: 47 (8.3\%) of the 564 studies included cited a published DES computer model; rising to 9.0\% in 2022. Studies were more likely to share models if they had been developed using free and open source tools. Studies rarely followed best practice when sharing computer models.Conclusions: Although still in the minority, there is evidence that healthcare DES authors are increasingly sharing their computer model artifacts. Although commercial software dominates the DES literature, free and open source software plays a crucial role in sharing. The DES community can adopt many simple best practices to improve the quality of sharing.

BackgroundWe aimed to select and externally validate a benchmark method for emergency ambulance services to use to forecast the daily number of calls that result in the dispatch of one or more ambulances. The study was conducted using standard methods known to the UK's NHS to aid implementation in practice.MethodsWe selected our benchmark model from a naive benchmark and 14 standard forecasting methods. Mean absolute scaled error and 80 and 95\% prediction interval coverage over a 84 day horizon were evaluated using time series cross validation across eight time series from the South West of England. External validation was conducted by time series cross validation across 13 time series from London, Yorkshire and Welsh Ambulance Services. ResultsA model combining a simple average of Facebook's Prophet and regression with ARIMA Errors (1, 1, 3)(1, 0, 1, 7) was selected. Benchmark MASE, 80 and 95\% prediction intervals were 0.68 (95% CI 0.67 - 0.69), 0.847 (95% CI 0.843 - 0.851), and 0.965 (95% CI 0.949 - 0.977), respectively. Performance in the validation set was within expected ranges for MASE, 0.73 (95% CI 0.72 - 0.74) 80\% coverage (0.833; 95% CI 0.828-0.838), and 95\% coverage (0.965; 95% CI 0.963-0.967).ConclusionsWe provide a robust externally validated benchmark for future ambulance demand forecasting studies to improve on. Our benchmark forecasting model is high quality and usable by ambulance services. We provide a simple python framework to aid its implementation in practice.

This paper provides a framework for conceptualising levels of open science and open working within computer modelling and simulation. We aim to support researchers to share their models and working so that others are free to use, reproduce, adapt and build upon, and re-share their work. We introduce a six level framework of increasing complexity: not open, open access, open artefacts, open models, open environment and open infrastructure. For each we provide practical advice on what aspects of open science researchers must consider, what options are available to them, and what challenges they will need to overcome. We illustrate our open science framework using a stylised discrete-event simulation model. All code used in this paper is available, cloud executable and reusable under an MIT license.

ABSTRACTObjectivesTo develop a simulation model to support orthopaedic elective capacity planning.MethodsAn open-source, generalisable discrete-event simulation was developed, including a web-based application. The model used anonymised patient records between 2016-2019 of elective orthopaedic procedures from an NHS Trust in England. In this paper, it is used to investigate scenarios including resourcing (beds and theatres) and productivity (lengths-of-stay, delayed discharges, theatre activity) to support planning for meeting new NHS targets aimed at reducing elective orthopaedic surgical backlogs in a proposed ring fenced orthopaedic surgical facility. The simulation is interactive and intended for use by health service planners and clinicians.ResultsA higher number of beds (65-70) than the proposed number (40 beds) will be required if lengths-of-stay and delayed discharge rates remain unchanged. Reducing lengths-of-stay in line with national benchmarks reduces bed utilisation to an estimated 60%, allowing for additional theatre activity such as weekend working. Further, reducing the proportion of patients with a delayed discharge by 75% reduces bed utilisation to below 40%, even with weekend working. A range of other scenarios can also be investigated directly by NHS planners using the interactive web app.ConclusionsThe simulation model is intended to support capacity planning of orthopaedic elective services by identifying a balance of capacity across theatres and beds and predicting the impact of productivity measures on capacity requirements. It is applicable beyond the study site and can be adapted for other specialties.Strengths and Limitations of this studyThe simulation model provides rapid quantitative estimates to support post-COVID elective services recovery toward medium-term elective targets.Parameter combinations include changes to both resourcing and productivity.The interactive web app enables intuitive parameter changes by users while underlying source code can be adapted or re-used for similar applications.Patient attributes such as complexity are not included in the model but are reflected in variables such as length-of-stay and delayed discharge rates.Theatre schedules are simplified, incorporating the five key orthopaedic elective surgical procedures.

Abstract
. Background
. We aimed to select and externally validate a benchmark method for emergency ambulance services to use to forecast the daily number of calls that result in the dispatch of one or more ambulances.
.
. Methods
. The study was conducted using standard methods known to the UK’s NHS to aid implementation in practice. We selected our benchmark model from a naive benchmark and 14 standard forecasting methods. Mean absolute scaled error and 80 and 95% prediction interval coverage over a 84 day horizon were evaluated using time series cross validation across eight time series from the South West of England. External validation was conducted by time series cross validation across 13 time series from London, Yorkshire and Welsh Ambulance Services.
.
. Results
. A model combining a simple average of Facebook’s prophet and regression with ARIMA errors (1, 1, 3)(1, 0, 1, 7) was selected. Benchmark MASE, 80 and 95% prediction intervals were 0.68 (95% CI 0.67 - 0.69), 0.847 (95% CI 0.843 - 0.851), and 0.965 (95% CI 0.949 - 0.977), respectively. Performance in the validation set was within expected ranges for MASE, 0.73 (95% CI 0.72 - 0.74) 80% coverage (0.833; 95% CI 0.828-0.838), and 95% coverage (0.965; 95% CI 0.963-0.967).
.
. Conclusions
. We provide a robust externally validated benchmark for future ambulance demand forecasting studies to improve on. Our benchmark forecasting model is high quality and usable by ambulance services. We provide a simple python framework to aid its implementation in practice. The results of this study were implemented in the South West of England.
.

BACKGROUND: it is a stated ambition of many healthcare systems to eliminate delayed transfers of care (DTOCs) between acute and step-down community services. OBJECTIVE: This study aims to demonstrate how, counter to intuition, pursual of such a policy is likely to be uneconomical, as it would require large amounts of community capacity to accommodate even the rarest of demand peaks, leaving much capacity unused for much of the time. METHODS: Some standard results from queueing theory-a mathematical discipline for considering the dynamics of queues and queueing systems-are used to provide a model of patient flow from the acute to community setting. While queueing models have a track record of application in healthcare, they have not before been used to address this question. RESULTS: Results show that 'eliminating' DTOCs is a false economy: the additional community costs required are greater than the possible acute cost saving. While a substantial proportion of DTOCs can be attributed to inefficient use of resources, the remainder can be considered economically essential to ensuring cost-efficient service operation. For England's National Health Service (NHS), our modelling estimates annual cost savings of £117m if DTOCs are reduced to the 12% of current levels that can be regarded as economically essential. CONCLUSION: This study discourages the use of 'zero DTOC' targets and instead supports an assessment based on the specific characteristics of the healthcare system considered.

OBJECTIVES: to identify risk factors associated with prolonged length of hospital stay and staying in hospital longer than medically necessary following primary knee replacement surgery. DESIGN: Retrospective, longitudinal observational study. SETTING: Elective knee replacement surgeries between 2016 and 2019 were identified using routinely collected data from an NHS Trust in England. PARTICIPANTS: There were 2295 knee replacement patients with complete data included in analysis. The mean age was 68 (SD 11) and 60% were female. OUTCOME MEASURES: We assessed a binary length of stay outcome (>7 days), a continuous length of stay outcome (≤30 days) and a binary measure of whether patients remained in hospital when they were medically fit for discharge. RESULTS: the mean length of stay was 5.0 days (SD 3.9), 15.4% of patients were in hospital for >7 days and 7.1% remained in hospital when they were medically fit for discharge. Longer length of stay was associated with older age (b=0.08, 95% CI 0.07 to 0.09), female sex (b=0.36, 95% CI 0.06 to 0.67), high deprivation (b=0.98, 95% CI 0.47 to 1.48) and more comorbidities (b=2.48, 95% CI 0.15 to 4.81). Remaining in hospital beyond being medically fit for discharge was associated with older age (OR=1.07, 95% CI 1.05 to 1.09), female sex (OR=1.71, 95% CI 1.19 to 2.47) and high deprivation (OR=2.27, 95% CI 1.27 to 4.06). CONCLUSIONS: the regression models could be used to identify which patients are likely to occupy hospital beds for longer. This could be helpful in scheduling operations to aid hospital efficiency by planning these patients' operations for when the hospital is less busy.

OBJECTIVES: While there has been significant research on the pressures facing acute hospitals during the COVID-19 pandemic, there has been less interest in downstream community services which have also been challenged in meeting demand. This study aimed to estimate the theoretical cost-optimal capacity requirement for 'step down' intermediate care services within a major healthcare system in England, at a time when considerable uncertainty remained regarding vaccination uptake and the easing of societal restrictions. METHODS: Demand for intermediate care was projected using an epidemiological model (for COVID-19 demand) and regressing upon public mobility (for non-COVID-19 demand). These were inputted to a computer simulation model of patient flow from acute discharge readiness to bedded and home-based Discharge to Assess (D2A) intermediate care services. Cost-optimal capacity was defined as that which yielded the lowest total cost of intermediate care provision and corresponding acute discharge delays. RESULTS: Increased intermediate care capacity is likely to bring about lower system-level costs, with the additional D2A investment more than offset by substantial reductions in costly acute discharge delays (leading also to improved patient outcome and experience). Results suggest that completely eliminating acute 'bed blocking' is unlikely economical (requiring large amounts of downstream capacity), and that health systems should instead target an appropriate tolerance based upon the specific characteristics of the pathway. CONCLUSIONS: Computer modelling can be a valuable asset for determining optimal capacity allocation along the complex care pathway. With results supporting a Business Case for increased downstream capacity, this study demonstrates how modelling can be applied in practice and provides a blueprint for use alongside the freely-available model code.

In healthcare, opportunities to use real-time data to support quick and effective decision-making are expanding rapidly, as data increases in volume, velocity and variety. In parallel, the need for short-term decision-support to improve system resilience is increasingly relevant, with the recent COVID-19 crisis underlining the pressure that our healthcare services are under to deliver safe, effective, quality care in the face of rapidly-shifting parameters.
A real-time hybrid model (HM) which combines real-time data, predictions, and simulation, has the potential to support short-term decision-making in healthcare. Considering decision-making as a consequence of situation awareness focuses the HM on what information is needed where, when, how, and by whom with a view toward sustained implementation. However the articulation between real-time decision-support tools and a sociotechnical approach to their development and implementation is currently lacking in the literature.
Having identified the need for a conceptual framework to support the development of real-time HMs for short-term decision-support, this research proposed and tested the Integrated Hybrid Analytics Framework (IHAF) through an examination of the stages of a Design Science methodology and insights from the literature examining decision-making in dynamic, sociotechnical systems, data analytics, and simulation. Informed by IHAF, a HM was developed using real-time Emergency Department data, time-series forecasting, and discrete-event simulation. The application started with patient questionnaires to support problem definition and to act as a formative evaluation, and was subsequently evaluated using staff interviews.
Evaluation of the application found multiple examples where the objectives of people or sub-systems are not aligned, resulting in inefficiencies and other quality problems, which are characteristic of complex adaptive sociotechnical systems. Synthesis of the literature, the formative evaluation, and the final evaluation found significant themes which can act as antecedents or evaluation criteria for future real-time HM studies in sociotechnical systems, in particular in healthcare. The generic utility of IHAF is emphasised for supporting future applications in similar domains.

The purpose of a modelling and simulation (M&S) study for real-world operations management applications is to support decision-making and inform potential action, therefore investigating the aspects of the modelling process which influence trust is important. Previous work has considered the question of trust through the lens of model validation. However, whilst a simulation model may be technically well executed, stakeholders’ trust in the results may also depend upon intangible factors such as interpersonal relationships. Existing literature has also focused on the credibility of the simulation practitioner, however the credibility attribute belongs to the stakeholder, and it ignores the trust aspects that may exist between the stakeholders and the model itself. In this paper, we argue that different facets of trust emerge throughout the stages of a simulation study, and both influence, and are influenced by, the interaction between the model, the modeller and the stakeholders of the study. We present a synthesis of existing literature and extend it by proposing a formative model of trust which presents a conceptualisation of this tripartite relationship. Our contribution is the identification of the different facets of trust in the lifecycle of a modelling and simulation study. We argue that these interacting facets converge via the three-way relationship between modeller, model and stakeholders toward epistemic trust in the knowledge generated by the simulation study and ultimately model acceptability and implementation. To the best of our knowledge, ours is the first study that focuses solely on the question of trust in an M&S study.