Import Other Formats
Convert individual tif/tiff, png and jpeg files into a stack
Note: IDCube directly imports only stacks. Import individual files requires converting individual files into a stack and them importing the formed stack.
Features:
IDCubePro® enables the user to convert multiple individual tiff, png and jpeg files into a stack (the user can select the type of a stack file: tiff, png or jpeg) and then import a stack file to IDCubePro®.
The following example Cells_tiffs dataset is used as an example: The file presents a compilation of 10 tiff files.
1. Click File → Import other formats and Convert to IDCube → Convert Multiple TIF/PNG/JPEG Files to Stack.
2. From the opened directory navigate to the folder and select individual tiff files that you would like to convert to a stack.
3. Press Open. A new window will be opened asking where to save the stack file. The default name is ‘Stack’ and the default extension is .tif.
You can rename the file and select a different type of file from the dropdown menu.
Click Save. The completion of the file conversion will be confirmed.
Press OK. The stack file will be ready for import into IDCubePro® as shown in the next chapter Import TIF/PNG/JPEG Stack.
Import stack of tif/tiff, png, and jpeg files
Features:
IDCube directly imports a stack with tif/tiff, png, and jpeg files. Cells_tiff_stack dataset is used as an example. To form a stack, see a previous section Convert individual TIF/PNG/JPEQ files to Stack.
Steps:
1. Click File → Import other formats and convert to IDCube → Import TIF/PNG/JPEG Stack.
2. Select a stack of tif/tiff files from the directory.
Click Open. The stack file will be converted to the IDCube format and ready to use for any other operations.
You can also save the tiff stack into the IDCube format using Save As… option from the File menu.
Import Raw/HDR and Dat/HDR files
Features:
raw/hdr and dat/hdr files are commonly used by commercial data acquisition systems. IDCubePro® enables the user to import and convert these types of files to the IDCube format
Hyperspectral datasets often use raw/hdr or dat/hdr format. These formats have at least two files.
The raw or dat files contains all relevant data and have a three dimensional structure (m x n x l), where m and n are spatial coordinates and l is the spectral coordinate.
The hdr is a header file that contains metadata. The header file typically contains information about the type of sensor used to acquire the image, the date of the dataset collection, the number of discrete spectral bands and the corresponding wavelengths or channels.
Typically, the header file uses the same name as the image file and is located in the same folder.
Steps:
1. Click File → Import other formats and Convert to IDCube → Import Raw/HDR or Dat/HDR.
2. Select a folder that has both files.
3. Select both dat and hdr files and click Open. The files will be opened in IDCubePro® and automatically converted to the IDCube format. After completion, metadata from the header file will be automatically shown. The file name will be preserved and saved in the original folder as shown below.
Import from different instruments
IDCubePro® supports many formats and platforms. For accessing data from satellites and other remote sensing platform, see our video tutorials on https://www.idcubes.com/tutorials.
The examples of using platforms from several type of platforms is given below.
Aviris
To find and download the file go to https://aviris.jpl.nasa.gov/ and click on the panel Search the Data Archive.
Aviris NG
AVIRIS NG stands for Aviris New Generation. AVIRIS-NG is an imaging spectrometer that measures reflected radiance at 5nm intervals in the Visible to Shortwave Infrared (VSWIR) spectral range from 380-2510nm.
Downloading data.
Notice: Aviris NG operates by a company/governmental agency that is not affiliated with IDCube. The access to the data and the quality of the data are not controlled be IDCubePro.
To find and download the file go https://avirisng.jpl.nasa.gov/data.html. Red areas indicate available data.
Find the data of your interest and click Download Orthocorrected Radiance Data. The download will start.
After the download is complete, unzip the file to see a folder with all files
2. To convert Aviris-NG files to IDCube, select Import Other Formats and Convert to IDCube → Aviris → Aviris NG
Navigate to the folder with the files and select an image datafile, usually the largest file and a header file that has the same name as the datafile but with extension .hdr. Click Open to initiate the conversion.
Completion of the conversion will be notified with a message box.
After the conversion is completed, the new imaging file in the IDCube format (with an .mat extension) will be generated and saved with the same name as the .L1R file.
Notice: AVIRIS-NG generates imaging data with values that are artificially low (highly negative). These data correspond to the edges of the image. If necessary, the best way to eliminate this data points is to crop the image.
Due to the presence of several header files, the header file will not be open automatically. It can be opened via View → View Header Information. Any file with an .hdr or .txt extension can be open this way.
Hyperion (tiff format)
Hyperion generates files in two formats with .tiff and .l1r/hdr extensions. In the case of tiff files, all tiff files are located within one folder. The folder has also a metadata file. To convert the tiff files to IDCube format, select Import Other Formats and Convert to IDCube → Hyperion.
A pop-up message will ask to select a folder where the tiff files are located.
Select the folder and click Select Folder where the tiff files are located. The conversion will start immediately. After the conversion is completed, the message will appear. The new imaging file in the IDCube format (with an .mat extension) will be generated and saved with the same name as the tiff files.
Hyperion (L1R/hdr format)
Hyperion generates files in two independent sets formats (tiff and L1R/hdr). This part of the manual describes how to download and process Hyperion files in the L1R/hdr format. The folder has also a metadata file.
Step 1: Selecting the area
Landsat 9 data can be obtained by visiting https://earthexplorer.usgs.gov/. You will have to login to the website in order to download the data.
Move to the location of interest and select the area by, for example, clicking on the map.
a. In the Data Range (below) specify the Date, Cloud Cover and other options.
b. In the Data Sets tab (on the top), go to EO-1 and select sat dropdown menu which contains EO-1 Hyperion.
When the download is complete, unzip the file.
Step 2. To import the file to IDCube format select File → Import Other Formats and Convert to IDCube → Hyperion (L1R/hdr).
A pop-up message will ask to select two files with .L1R (datafile) and .hdr (metadata) extensions.
Click OK and select two files. The Image data file is usually the largest file and the header file has an .hdr extension.
Select the files and click Open. The conversion will start immediately. After the conversion is completed, the header file will be opened automatically. The new imaging file in the IDCube format (with an .mat extension) will be generated and saved with the same name as the L1R file.
Sentinel 2
Sentinel-2 is an Earth observation mission from the Copernicus Programme that systematically acquires optical imagery at high spatial resolution over land and coastal waters. The mission is currently a constellation with two satellites, Sentinel-2A (launched in 2015) and Sentinel-2B (launched in 2017); a third satellite, Sentinel-2C, is currently undergoing testing in preparation for launch in 2024. Level-2A data is the most ideal for research activities as it allows further analysis without applying additional atmospheric corrections.
Sentinel 2 collects multi-spectral data with 13 bands in the visible (B02, B03, B04), near-infrared (B05, B06, B08, B08A), and short-wave infrared part (B09, B10, B11, B12) of the spectrum.
Step 1. Download data.
The data can be downloaded from several resources such as Copernicus Open Access Hub (https://scihub.copernicus.eu/dhus/#/home) and Earth Explore (https://earthexplorer.usgs.gov/).
A brief guidance from the Copernicus Open Access Hub is shown here.
Press the Open Hub panel and put your login information (or register if you are a new user). After that switch the Navigation Mode to Area Mode.
Then, 1) draw the region of interest (which will turn yellow). 2) Press the three lines on the Search Criteria and add criteria to narrow down the search. We suggest specifying Sentinel 2 as a mission platform. 3) Click Search to start the search process.
The search has identified 1045 datasets. Select the appropriate dataset and download it.
Unzip the file. Sentinel 2 dataset has the structure as shown on the left panel. Navigate to the IMG_DATA folder. In some cases, the IMG_DATA folder has subfolders, such as R10, R20, and R60. R20 and R60 are supported by the IDCubePro®. If any of the subfolders are selected (i.e., R20) you can see all bands that are available for processing. Refer to the meaning of each band in the Table of Sentinel 2 bands above.
Step 2. Converting data into the IDCube format.
Select Import Other Formats and Convert to IDCube → Sentinel 2.
You will be prompted to select the folder with the data that will be imported into the IDCube format.
After selecting the IMG_DATA folder, a waitbar will appear to show the conversion progress.
The converted file will be saved in the original folder/file location.
The converted file will automatically be opened and visualized in the IMAGE DISPLAY panel. In this example, the data in a folder containing Sentinel 2 images were converted to a single IDCube file, saved in the same directory and opened in the IDCube format.
Landsat 9
Features: IDCubePro® allows for the conversion of different file formats to IDCube files. This manual will outline the steps to download and convert Landsat 9 data from the USGS website to IDCube format files.
Step 1: Selecting the area and Landsat 9
Landsat 9 data can be obtained by visiting https://earthexplorer.usgs.gov/. You will have to login to the website in order to download the data.
Move to the location of interest and select the area by, for example, clicking on the map
c. In the Data Range (below) specify the Date, Cloud Cover and other options.
d. In the Data Sets tab (on the top), find a Landsat dropdown menu which contains “Landsat Collection 2 Level 1” data. In this dropdown menu, the “Landsat 8-9” data can be selected.
e. Any dataset beginning with “LC09…” can then be selected and downloaded.
Step 2. Downloading the data
a. The “Product Bundle” must be selected so that the entire dataset can be converted.
Once downloaded the dataset will appear as a zip file. This file can be unzipped using any software. We use 7-Zip software to extract the data. The resulting data file will be a folder.
Step 3. Converting the data from the Landsat to IDCube format.
The conversion function in IDCube can be accessed by navigating to the File menu → Import Other Formats and Convert to IDCube → Landsat → Landsat 9.
The desired Landsat 9 data folder can then be selected for conversion.
The converted file will be saved in the original folder path and the file will automatically be opened in IDCube.
