Pressemeddelelse
VLT Data Flow System Begins Operation
Building a Terabyte Archive at the ESO Headquarters
21. juni 1999
The site at Cerro Paranal in the dry Atacama desert in Northern Chile is one of the best locations for astronomical observations from the surface of the Earth. Each of the four 8.2-m telescopes is a technological marvel with self-adjusting optics placed in a gigantic mechanical structure of the utmost precision, continuously controlled by advanced soft- and hardware. A multitude of extremely complex instruments with sensitive detectors capture the faint light from distant objects in the Universe and record the digital data fast and efficiently as images and spectra, with a minimum of induced noise. And now the next crucial link in this chain is in place. A few nights ago, following an extended test period, the VLT Data Flow System began providing the astronomers with a steady stream of high-quality, calibrated image and spectral data, ready to be interpreted. The VLT project has entered into a new phase with a larger degree of automation. Indeed, the first 8.2-m Unit Telescope, ANTU, with the FORS1 and ISAAC instruments, has now become a true astronomy machine.
A smooth flow of data through the entire system
Already from the start of this project more than ten years ago, the ESO Very Large Telescope was conceived as a complex digital facility to explore the Universe. In order for astronomers to be able to use this marvellous research tool in the most efficient manner possible, the VLT computer software and hardware systems must guarantee a smooth flow of scientific information through the entire system.
This process starts when the astronomers submit well-considered proposals for observing time and it ends with large volumes of valuable astronomical data being distributed to the international astronomical community. For this, ESO has produced an integrated collection of software and hardware, known as the VLT Data Flow System (DFS) , that manages and facilitates the flow of scientific information within the VLT Observatory. Early information about this new concept was published as eso9612 and extensive tests were first carried out at ESOs 3.5-m New Technology Telescope (NTT) at La Silla, cf. eso9703 [1].
The VLT DFS is a complete (end-to-end) system that guarantees the highest data quality by optimization of the observing process and repeated checks that identify and eliminate any problems. It also introduces automatic calibration of the data, i.e. the removal of external effects introduced by the atmospheric conditions at the time of the observations, as well as the momentary state of the telescope and the instruments.
From Proposals to Observations
In order to obtain observing time with ESO telescopes, also with the VLT, astronomers must submit a detailed observing proposal to the ESO Observing Programmes Committee (OPC) . It meets twice a year and ranks the proposals according to scientific merit. More than 1000 proposals are submitted each year, mostly by astronomers from the ESO members states and Chile; the competition is fierce and only a fraction of the total demand for observing time can be fulfilled.
During the submission of observing proposals, DFS software tools available over the World Wide Web enable the astronomers to simulate their proposed observations and provide accurate estimates of the amount of telescope time they will need to complete their particular scientific programme.
Once the proposals have been reviewed by the OPC and telescope time is awarded by the ESO management according to the recommendation by this Committee, the successful astronomers begin to assemble detailed descriptions of their intended observations (e.g. position in the sky, time and duration of the observation, the instrument mode, etc.) in the form of computer files called Observation Blocks (OBs) . The software to make OBs is distributed by ESO and used by the astronomers at their home institutions to design their observing programs well before the observations are scheduled at the telescope.
The OBs can then be directly executed by the VLT and result in an increased efficiency in the collection of raw data (images, spectra) from the science instruments on the VLT. The activation (execution) of OBs can be done by the astronomer at the telescope on a particular set of dates (visitor mode operation) or it can be done by ESO science operations astronomers at times which are optimally suited for the particular scientific programme (service mode operation).
An enormous VLT Data Archive
The raw data generated at the telescope are stored by an archive system that sends these data regularly back to ESO headquarters in Garching (Germany) in the form of CD and DVD ROM disks. While the well-known Compact Disks (CD ROMs) store about 600 Megabytes (600,000,000 bytes) each, the new Digital Versatile Disks (DVD ROMs) - of the same physical size - can store up 3.9 Gigabytes (3,900,000,000 bytes) each, or over 6 times more.
The VLT will eventually produce more than 20 Gigabytes (20,000,000,000 bytes) of astronomical data every night, corresponding to about 10 million pages of text [2]. Some of these data also pass through "software pipelines" that automatically remove the instrumental effects on the data and deliver data products to the astronomer that can more readily be turned into scientific results.
Ultimately these data are stored in a permanent Science Archive Facility at ESO headquarters which is jointly operated by ESO and the Space Telescope European Coordinating Facility (ST-ECF). From here, data are distributed to astronomers on CD ROMs and over the World Wide Web. The archive facility is being developed to enable astronomers to "mine" the large volumes of data that will be collected from the VLT in the coming years. Within the first five years of operations the VLT is expected to produce around 100 Terabytes (100,000,000,000,000 bytes) of data.
It is difficult to visualize this enormous amount of information. However, it corresponds to the content of 50 million books of 1000 pages each; they would occupy some 2,500 kilometres of bookshelves!
The VLT Data Flow System enters into operation
Science operations with the first VLT 8.2-m telescope (ANTU) began on April 1, 1999. Following the first call for proposals to use the VLT in October 1998, the OPC met in December and the observing schedule was finalized early 1999. The related Observation Blocks were prepared by the astronomers in February and March. Service-mode observations began in April and by late May the first scientific programs conducted by ESO science operations were completed.
Raw data, instrument calibration information and the products of pipeline processing from these programs have now been assembled and packed onto CD ROMs by ESO science operations staff. On June 15 the first CD ROMs were delivered to astronomers in the ESO community.
This event marks the closing of the data flow loop at the VLT for the first time and the successful culmination of more than 5 years of hard work by ESO engineers and scientists to implement a system for efficient and effective scientific data flow. This was achieved by a cross-organization science operations team involving staff in Chile and Europe.
With the VLT Data Flow System, a wider research community will have access to the enormous wealth of data from the VLT. It will help astronomers to keep pace with the new technologies and extensive capabilities of the VLT and so obtain world-first scientific results and new insights into the universe.
Noter
[1] A more technical description of the VLT Data Flow System is available in Chapter 10 of the VLT Whitebook.
[2] By definition, one "normal printed page" contains 2,000 characters.
Om pressemeddelelsen
Pressemeddelelse nr.: | eso9930 |
Legacy ID: | PR 10/99 |
Navn: | ESO Data Centre |
Type: | Unspecified : Technology : Observatory : Facility Unspecified : Technology : Observatory : Telescope |
Facility: | Very Large Telescope |
Our use of Cookies
We use cookies that are essential for accessing our websites and using our services. We also use cookies to analyse, measure and improve our websites’ performance, to enable content sharing via social media and to display media content hosted on third-party platforms.
ESO Cookies Policy
The European Organisation for Astronomical Research in the Southern Hemisphere (ESO) is the pre-eminent intergovernmental science and technology organisation in astronomy. It carries out an ambitious programme focused on the design, construction and operation of powerful ground-based observing facilities for astronomy.
This Cookies Policy is intended to provide clarity by outlining the cookies used on the ESO public websites, their functions, the options you have for controlling them, and the ways you can contact us for additional details.
What are cookies?
Cookies are small pieces of data stored on your device by websites you visit. They serve various purposes, such as remembering login credentials and preferences and enhance your browsing experience.
Categories of cookies we use
Essential cookies (always active): These cookies are strictly necessary for the proper functioning of our website. Without these cookies, the website cannot operate correctly, and certain services, such as logging in or accessing secure areas, may not be available; because they are essential for the website’s operation, they cannot be disabled.
Functional Cookies: These cookies enhance your browsing experience by enabling additional features and personalization, such as remembering your preferences and settings. While not strictly necessary for the website to function, they improve usability and convenience; these cookies are only placed if you provide your consent.
Analytics cookies: These cookies collect information about how visitors interact with our website, such as which pages are visited most often and how users navigate the site. This data helps us improve website performance, optimize content, and enhance the user experience; these cookies are only placed if you provide your consent. We use the following analytics cookies.
Matomo Cookies:
This website uses Matomo (formerly Piwik), an open source software which enables the statistical analysis of website visits. Matomo uses cookies (text files) which are saved on your computer and which allow us to analyze how you use our website. The website user information generated by the cookies will only be saved on the servers of our IT Department. We use this information to analyze www.eso.org visits and to prepare reports on website activities. These data will not be disclosed to third parties.
On behalf of ESO, Matomo will use this information for the purpose of evaluating your use of the website, compiling reports on website activity and providing other services relating to website activity and internet usage.
Matomo cookies settings:
Additional Third-party cookies on ESO websites: some of our pages display content from external providers, e.g. YouTube.
Such third-party services are outside of ESO control and may, at any time, change their terms of service, use of cookies, etc.
YouTube: Some videos on the ESO website are embedded from ESO’s official YouTube channel. We have enabled YouTube’s privacy-enhanced mode, meaning that no cookies are set unless the user actively clicks on the video to play it. Additionally, in this mode, YouTube does not store any personally identifiable cookie data for embedded video playbacks. For more details, please refer to YouTube’s embedding videos information page.
Cookies can also be classified based on the following elements.
Regarding the domain, there are:
- First-party cookies, set by the website you are currently visiting. They are stored by the same domain that you are browsing and are used to enhance your experience on that site;
- Third-party cookies, set by a domain other than the one you are currently visiting.
As for their duration, cookies can be:
- Browser-session cookies, which are deleted when the user closes the browser;
- Stored cookies, which stay on the user's device for a predetermined period of time.
How to manage cookies
Cookie settings: You can modify your cookie choices for the ESO webpages at any time by clicking on the link Cookie settings at the bottom of any page.
In your browser: If you wish to delete cookies or instruct your browser to delete or block cookies by default, please visit the help pages of your browser:
Please be aware that if you delete or decline cookies, certain functionalities of our website may be not be available and your browsing experience may be affected.
You can set most browsers to prevent any cookies being placed on your device, but you may then have to manually adjust some preferences every time you visit a site/page. And some services and functionalities may not work properly at all (e.g. profile logging-in, shop check out).
Updates to the ESO Cookies Policy
The ESO Cookies Policy may be subject to future updates, which will be made available on this page.
Additional information
For any queries related to cookies, please contact: pdprATesoDOTorg.
As ESO public webpages are managed by our Department of Communication, your questions will be dealt with the support of the said Department.