Table 4 Formative evaluation (FE) measures and evaluation schedule

Reach/Implementation Analysis

Referral rates

Number of patients referred on weekly and monthly basis.

Treatment uptake

Percent of referred patients enrolling in treatment, engaging in treatment, and completing treatment.

Exposure to TeleQuitMH

Percent of MH patients for whom tobacco use screening was completed.

Patient Process Data

Enrollment Rates

Rates of enrollment to TeleQuit MH and State quitlines

TeleQuitMH Satisfaction

Self-reported satisfaction with TeleQuitMH structure and treatment (including VA versus quitline satisfaction) ; assessed 2m and 6m after enrolling

Provider Process Data

Provider Survey

Perceptions of program marketing; how likely they will be to use the program; barriers to implementation; suggestions for improvement; provider use of the program; program satisfaction; perceived effectiveness of activities to facilitate implementation; impact of the program on their patients/client.

Stakeholder Interviews

Perceptions of program marketing; how likely they will be to use the program; barriers to implementation; suggestions for improvement; provider use of the program; likes/dislikes about the program; impact of the program on their patients/clients.

Referral rates/patterns

Percent of providers referring; Number of referrals from each provider; Rates by provider type

MH Unit Data on Implementation Context

VA Survey of MH Programs[28]

Clinic’s structure, size, staffing, environment, and resources for meeting MH and tobacco cessation treatment goals.

MH Unit Performance Data[29]

Clinic-level performance on VA MH and tobacco performance measures

Implementation Facilitation - Structure

Site Champion

Presence of a site champion, level of interest in performing site champion duties, position at the VA, role on the project

Site Clinical Advisory Committee

Presence of a site Clinical Advisory Committee, level of interest in performing Committee duties, Committee members positions at the VA & roles on the project

Site smoking cessation services

Presence, types, and structure of site smoking cessation clinic and other local smoking cessation services

TeleQuitMH Structure

Structure of the TeleQuitMH program at each site, including design and appearance of the tobacco cessation clinical reminder, prescribing structure, and referral mechanisms (e.g., consult design, email referral option)

State quitline Structure/Issues

Structure of quitline services; issues/problems identified in connecting TeleQuitMH patients to quitline services.

IRB Issues

Time from first submission to receiving IRB approval; type of submission required (full board review, expedited, exempt); site-specific requirements that may affect TeleQuitMH implementation and adoption (e.g., detailed consent process)

Implementation Facilitation - Activities

Marketing Efforts

Provider outreach efforts, academic detailing, provider feedback.

Treatment Fidelity

Extent to which the TeleQuitMH staff are providing the telephone counseling, medication coordination and warm-transfer to the state quitlines as intended.