Once per year, now in the third quarter of the year, the User Support Department of ESO launches a Paranal Service Mode User Satisfaction Survey campaign.  This report details the findings of the September 2017 survey campaign, while previous such reports are found here. Since we now have 4 previous surveys made with the same methodology, we are in a position to show trending plots in addition to statistics for the given year.

We view these reports as an important way to close the loop with the ESO Community, to gather information on issues that need to be addressed or re-inforced, to thank all respondents, and to demonstrate that such feedback is important to us. To this end, here we provide a summary of the responses received, predominantly in the form of graphs. It should also be stressed that for those cases where respondents did identify themselves and did make specific free-text comments we have contacted them by e-mail to address their particular comments.

Methodology and General Results

The ESO Service Mode Questionnaire is always available on-line for users to fill in but the usual rate of return is less than 2 per month. However, experience shows that a targeted campaign focused on a single (in this case Phase 2 related) aspect results in many more survey completions.

In September 2017, we again took this approach, and asked Principal Investigators (PIs) of Service Mode runs scheduled for Paranal in Periods 99 and/or 100¹ (plus their then-active Phase 2 delegates) to complete the survey by a fixed deadline. We thus solicited a response from 504 PIs and their then-active Phase 2 delegates (143 individuals). Because of overlap this amounts to a total of 580 individuals which were contacted via e-mail. A deadline was set for two weeks from the date of contact.

A total of 177 responses were received by the deadline, at which time the deadline was extended for an additional week.  After that additional week the number of submissions had risen to a total of 194 (some 13 of which were not fully complete), representing a 33.4% response rate (see the figure below). Here we note that as of 2017 we have slightly refined handling of incomplete responses. 

Interactive Figure Features

The figures below are all interactive.  By this we mean:

• Puttling the cursor over the plot will display the data values on the screen.
• Clicking on the menu icon in the upper right (the three short parallel horizontal lines) will open a menu of print/download options.
• For those figures with legends to the right of the plot clicking on any entry in the legend will toggle display of the corresponding data within the graph.
• Clicking on the Linux wedge of the final plot will show the breakdown by distribution.  

As a start in detailing the results from the survey, in the figure below we show the number of responses we received per instrument. In spite of the overall good response rate the large number of instruments offered in Service Mode means that on average this year we received about 15.5 responses per instrument.

In the following three stacked histogram plots we present a general overview of user satisfaction (in percentage of responses) with three general items:

• the help/advice provided by the User Support Department during the Phase 2 process
• the Phase 2 web documentation, and
• the overall support provided by the User Support Department.

Note that the sum of the responses to the question about one’s satisfaction with the help/advice provided is less than the total number of survey responses. However, there is no reason to expect, a priori, that these two numbers should be equal. This is because the responses to this question come from a subset of respondents (those that actually received help/advice at Phase 2), each of which may have received help/advice from multiple instruments.

The plots, designed to show the trend in user satisfaction over the last five surveys, clearly show the consistently high satisfaction with these services offered by the User Support Department.

Seeking Help, Run Information, and Run Problem Resolution

Amongst the respondents 24.2% indicated that they had contacted ESO for non-Phase 2 related reasons within the previous 12 months (about the same percentage as in the September 2016 survey). Of these 47 respondents, 40 contacted ESO via an e-mail to usd-help@eso.org (again consistent with September 2016), with the remainder distributed between other methods (e.g. clicking on ‘Ask for help’ within the ESO User Portal).  Below we show the degree to which these respondents were satisfied with various aspects of the resulting exchange with ESO, as above with the results of the previous 4 surveys for comparison.  As above, the satisfaction level is consistently high.

Some 76% of the respondents checked on-line for information regarding the progress of their observational programmes (this represents an increase over the 71% of respondents that reported having checked this information in September 2016).  The survey asked those that did check for that information how much they agreed with four statements about that information. The outcome of those questions is shown below also displaying the results of the previous 4 surveys for comparison.  As in the past, with a few exceptonal cases the presentation of this information was deemed easily locatable, clearly presented, up-to-date, and complete.

Finally, we asked if the survey participant was alerted to any problems with their Service Mode observations during the period. For those that said that they had been contacted (48, a slight drop for the 53 in September 2016) we then asked to what extent they agreed with two statements describing the contact and the problem resolution. The answers are shown below.  As in the past, with a few exceptonal cases the problem was clearly described and a fair solution achieved.

P2PP and Other Observation Preparation Tools

Below, we show details of the feedback received on different aspects of the Phase 2 Proposal Preparation tool (P2PP) and other, instrument-specific, observation preparation tools, with the previous 4 survey results for comparative purposes.  The satisfaction level is somewhat poorer with this tool that with the above mentioned services frovided by the User Support Department.

Since the number of responses per observing preparation tool other than P2PP is rather limited (see the table below), any presentation of individual-tool responses on documentation, ease of use, or functionality would suffer from small number statistics. Thus, in the figures below answers for all tools are combined.  As with P2PP, when one considers the other tools as an emsemble we see room for improvement.

Related to the above tools is, of course, the suite of Exposure Time Calculators.  Thus, we asked survey participants the question, “How satisfied are you with the ETCs you have used?” The responses are shown below.  There are consistently few respondents that express dissatisfaction with the ETC.  The ~8% (average) of "No opinion" answers could be interpreted as that fraction of the respondents that did not use any ETC.

And lastly we asked survey participants, "Which operating system(s) do you use for ESO tools (e.g. for proposal/observation preparation, data reduction), excluding any browser-based tools?"  The breakdown of responses is shown in the interactive figure below.

Notes:

¹The total time allocated in Service Mode for Periods 99 and 100 was 12388.35 hours, while the corresponding number for Visitor Mode was 4517.3 hours.  Thus, the September 2017 survey targetted PIs (and their then-active delegates) representing 73.2% of the total VLT/VLTI time allocation, including all public surveys.