NCDatasets allows one to read and create netCDF files.
NetCDF data set and attribute list behave like Julia dictionaries and variables like Julia arrays.
NCDatasets provides support for the following netCDF CF conventions:
_FillValuewill be returned as
add_offsetare applied if present
unitsattribute) are returned as
Other features include:
ncgenwhich generates the Julia code that would produce a netCDF file with the same metadata as a template netCDF file.
Inside the Julia shell, you can download and install the package by issuing:
If you want to try the latest development version, you can do this with the following commands:
Before reading the data from a netCDF file, it is often useful to explore the list of variables and attributes defined in it.
For interactive use, the following commands (without ending semicolon) display the content of the file similarly to
ncdump -h file.nc
using NCDatasets ds = Dataset("file.nc")
The following displays the information just for the variable
varname and for the global attributes:
The following gives an example of how to create a netCDF file by defining dimensions, variables and attributes.
using NCDatasets # This creates a new NetCDF file /tmp/test.nc. # The mode "c" stands for creating a new file (clobber) ds = Dataset("/tmp/test.nc","c") # Define the dimension "lon" and "lat" with the size 100 and 110 resp. defDim(ds,"lon",100) defDim(ds,"lat",110) # Define a global attribute ds.attrib["title"] = "this is a test file" # Define the variables temperature v = defVar(ds,"temperature",Float32,("lon","lat")) # Generate some example data data = [Float32(i+j) for i = 1:100, j = 1:110] # write a single column v[:,1] = data[:,1] # write a the complete data set v[:,:] = data # write attributes v.attrib["units"] = "degree Celsius" v.attrib["comments"] = "this is a string attribute with Unicode Ω ∈ ∑ ∫ f(x) dx" close(ds)
An equivalent way to create the previous NetCDF would be the following code:
using NCDatasets data = [Float32(i+j) for i = 1:100, j = 1:110] Dataset("/tmp/test2.nc","c",attrib = ["title" => "this is a test file"]) do ds # Define the variable temperature. The dimension "lon" and "lat" with the # size 100 and 110 resp are implicetly created defVar(ds,"temperature",data,("lon","lat"), attrib = [ "units" => "degree Celsius", "comments" => "this is a string attribute with Unicode Ω ∈ ∑ ∫ f(x) dx" ]) end
# download example file ncfile = download("https://www.unidata.ucar.edu/software/netcdf/examples/sresa1b_ncar_ccsm3-example.nc") # generate Julia code ncgen(ncfile)
The produces the Julia code (only the beginning of the code is shown):
ds = Dataset("filename.nc","c") # Dimensions ds.dim["lat"] = 128; ds.dim["lon"] = 256; ds.dim["bnds"] = 2; ds.dim["plev"] = 17; ds.dim["time"] = 1; # Declare variables ncarea = defVar(ds,"area", Float32, ("lon", "lat")) ncarea.attrib["long_name"] = "Surface area"; ncarea.attrib["units"] = "meter2"; # ...
Loading a variable with known structure can be achieved by accessing the variables and attributes directly by their name.
# The mode "r" stands for read-only. The mode "r" is the default mode and the parameter can be omitted. ds = Dataset("/tmp/test.nc","r") v = ds["temperature"] # load a subset subdata = v[10:30,30:5:end] # load all data data = v[:,:] # load all data ignoring attributes like scale_factor, add_offset, _FillValue and time units data2 = v.var[:,:] # load an attribute unit = v.attrib["units"] close(ds)
In the example above, the subset can also be loaded with:
subdata = Dataset("/tmp/test.nc")["temperature"][10:30,30:5:end]
This might be useful in an interactive session. However, the file
test.nc is not closed, which can be a problem if you open many files. On Linux the number of opened files is often limited to 1024 (soft limit). If you write to a file, you should also always close the file to make sure that the data is properly written to the disk.
An alternative way to ensure the file has been closed is to use a
do block: the file will be closed automatically when leaving the block.
Dataset(filename,"r") do ds data = ds["temperature"][:,:] end # ds is closed
If the structure of the netCDF file is not known before-hand, the program must check if a variable or attribute exists (with the
in operator) before loading it or alternatively place the loading in a
It is also possible to iterate over all variables or attributes (global attributes or variable attributes) in the same syntax as iterating over a dictionary. However, unlike Julia dictionaries, the order of the attributes and variables is preserved and presented as they are stored in the netCDF file.
# Open a file as read-only ds = Dataset("/tmp/test.nc","r") # check if a file has a variable with a given name if haskey(ds,"temperature") println("The file has a variable 'temperature'") end # get a list of all variable names @show keys(ds) # iterate over all variables for (varname,var) in ds @show (varname,size(var)) end # query size of a variable (without loading it) v = ds["temperature"] @show size(v) # similar for global and variable attributes if haskey(ds.attrib,"title") println("The file has the global attribute 'title'") end # get an list of all attribute names @show keys(ds.attrib) # iterate over all attributes for (attname,attval) in ds.attrib @show (attname,attval) end # get the attribute "units" of the variable v # but return the default value (here "adimensional") # if the attribute does not exists units = get(v,"units","adimensional") close(ds)
The variable name are not always standardized, for example the longitude we can
The solution implemented in the function
varbyattrib consists in searching for the
variables that have specified value for a given attribute.
lon = varbyattrib(ds, standard_name="longitude");
will return the list of variables of the dataset
ds that have "longitude"
as standard name.
When you file an issue, please include sufficient information that would allow somebody else to reproduce the issue, in particular:
NCDatasetsare you using? Please include the output of:
html versioninfo() Pkg.installed()["NCDatasets"]
NCDatasetspass its test suite? Please include the output of:
The package NetCDF.jl from Fabian Gans and contributors is an alternative to this package which supports a more Matlab/Octave-like interface for reading and writing NetCDF files.
build.jl and the error handling code of the NetCDF C API are from NetCDF.jl by Fabian Gans (Max-Planck-Institut für Biogeochemie, Jena, Germany) released under the MIT license.
3 days ago