API Reference¶
Opening/copying data¶
|
Read netCDF data as a DataSet object |
|
Read netCDF data from a url as a DataSet object |
|
Read thredds data as a DataSet object |
|
Read geotiff and convert to nctoolkit dataset |
|
Convert an xarray dataset to an nctoolkit dataset This will first save the xarray dataset as a temporary netCDF file. |
|
Make a deep copy of an DataSet object Note: This will not make disk copies of the temporary files underlying datasets, so it will be disk-space efficient. |
Merging or analyzing multiple datasets¶
|
Merge datasets |
|
Calculate the temporal correlation coefficient between two datasets This will calculate the temporal correlation coefficient, for each time step, between two datasets. |
|
Calculate the spatial correlation coefficient between two datasets This will calculate the spatial correlation coefficient, for each time step, between two datasets. |
Accessing attributes¶
List variables contained in a dataset |
|
Detailed list of variables contained in a dataset. |
|
List times contained in a dataset |
|
List years contained in a dataset |
|
List months contained in a dataset |
|
List levels contained in a dataset |
|
The size of an object This will print the number of files, total size, and smallest and largest files in an DataSet object. |
|
The current file or files in the DataSet object |
|
The history of operations on the DataSet |
|
The starting file or files of the DataSet object |
|
List calendars of dataset files |
|
List formats of files contained in a dataset |
Plotting¶
|
Variable modification¶
|
Create new variables Existing columns that are re-assigned will be overwritten. :param drop: Set to True if you want existing variables to be removed once the new ones have been created. Defaults to False. |
|
Rename variables in a dataset |
|
Set the missing value for a single number or a range |
|
Calculate the sum of all variables for each time step |
netCDF file attribute modification¶
|
Set the long names of variables |
|
Set the units for variables |
Vertical/level methods¶
|
Extract the top/surface level from a dataset This extracts the first vertical level from each file in a dataset. |
|
Extract the bottom level from a dataset This extracts the bottom level from each netCDF file. |
|
Verticaly interpolate a dataset based on given vertical levels This is calculated for each time step and grid cell |
|
Calculate the depth-averaged mean for each variable This is calculated for each time step and grid cell |
|
Calculate the vertical minimum of variable values This is calculated for each time step and grid cell |
|
Calculate the vertical maximum of variable values This is calculated for each time step and grid cell |
|
Calculate the vertical range of variable values This is calculated for each time step and grid cell |
|
Calculate the vertical sum of variable values This is calculated for each time step and grid cell |
|
Calculate the vertically integrated sum over the water column This calculates the sum of the variable multiplied by the cell thickness |
|
Calculate the vertical sum of variable values This is calculated for each time step and grid cell |
|
Invert the levels of 3D variables This is calculated for each time step and grid cell |
|
Create a mask identifying the deepest cell without missing values. |
Rolling methods¶
|
Calculate a rolling mean based on a window Time output is the middle time of the window. |
|
Calculate a rolling minimum based on a window Time output is the middle time of the window. |
|
Calculate a rolling maximum based on a window Time output is the middle time of the window. |
|
Calculate a rolling sum based on a window Time output is the middle time of the window. |
|
Calculate a rolling range based on a window Time output is the middle time of the window. |
Evaluation setting¶
|
Run all stored commands in a dataset |
Cleaning functions¶
Ensemble creation¶
|
Generate an ensemble |
Arithemetic methods¶
|
Method to get the absolute value of variables |
|
Add to a dataset This will add a constant, another dataset or a netCDF file to the dataset. |
|
Create new variables Existing columns that are re-assigned will be overwritten. :param drop: Set to True if you want existing variables to be removed once the new ones have been created. Defaults to False. |
|
Method to get the exponential of variables |
|
Method to get the natural log of variables |
|
Method to get the base 10 log of variables |
|
Multiply a dataset This will multiply a dataset by a constant, another dataset or a netCDF file. :param x: An int, float, single file dataset or netCDF file to multiply the dataset by. If multiplying by a dataset or single file there must only be a single variable in it, unless var is supplied. The grids must be the same. :type x: int, float, DataSet or netCDF file :param var: A variable in the x to multiply the dataset by :type var: str. |
|
Powers of variables in dataset :param x: An int or float to take the variables to the power of :type x: int, float |
|
Method to get the square root of variables |
|
Method to get the square of variables |
|
Subtract from a dataset This will subtract a constant, another dataset or a netCDF file from the dataset. :param x: An int, float, single file dataset or netCDF file to subtract from the dataset. If a dataset or netCDF is supplied this must only have one variable, unless var is provided. The grids must be the same. :type x: int, float, DataSet or netCDF file :param var: A variable in the x to use for the operation :type var: str. |
|
Divide the data This will divide the dataset by a constant, another dataset or a netCDF file. :param x: An int, float, single file dataset or netCDF file to divide the dataset by. If a dataset or netCDF file is supplied, this must have only one variable, unless var is provided. The grids must be the same. :type x: int, float, DataSet or netCDF file :param var: A variable in the x to use for the operation :type var: str. |
Ensemble statistics¶
|
Calculate an ensemble mean |
|
Calculate an ensemble min |
|
Calculate an ensemble maximum |
|
Calculate an ensemble percentile This will calculate the percentles for each time step in the files. |
|
Calculate an ensemble range The range is calculated for each time step; for example, if each file in the ensemble has 12 months of data the statistic will be calculated for each month. |
|
Calculate an ensemble standard deviation |
|
Calculate an ensemble sum The sum is calculated for each time step; for example, if each file in the ensemble has 12 months of data the statistic will be calculated for each month. |
|
Calculate an ensemble variance |
Subsetting operations¶
|
Crop to a rectangular longitude and latitude box |
|
A method for subsetting datasets to specific variables, years, longitudes etc. |
|
Remove variables This will remove stated variables from files in the dataset. |
Time-based methods¶
|
Set the date in a dataset You should only do this if you have to fix/change a dataset with a single, not multiple dates. |
|
Shift method. |
Interpolation, matching and resampling methods¶
|
Regrid a dataset to a target grid |
|
Regrid a dataset to a regular latlon grid |
|
Match dataset to a spatiotemporal points dataframe :param df: The column names must be made up of a subset of “lon”, “lat”, “year”, “month”, “day” and “depth” :type df: pandas dataframe containing the spatiotemporal points to match with. :param variables: Str or list of variables. All variables are matched up if this is not supplied. :type variables: str or list :param depths: If each cell has different vertical levels, this must be provided as a dataset. If each cell has the same vertical levels, provide it as a list. If this is not supplied nctoolkit will try to figure out what they are. Only required if carrying out vertical matchups. :type depths: nctoolkit dataset or list giving depths :param tmean: Set to True or False, depending on whether you want temporal averaging at the temporal resolution given by df. For example, if you only had months in df, but had daily data in ds, you might want to calculate a daily average in the monthly dataset. This is equivalent to apply ds.tmean(..) to the dataset. :type tmean: bool :param top: Set to True if you want only the top/surface level of the dataset to be selected for matching. :type top: bool :param nan: Value or range of values to set to nan. Defaults to 0. Only required if values in dataset need changed to missing :type nan: float or list :param regrid: Regridding method. Defaults to “bil”. Options available are those in nctoolkit regrid method. “nn” for nearest neighbour. :type regrid: str. |
|
Resample the horizontal grid of a dataset |
|
Temporally interpolate variables based on date range and time resolution |
|
Temporally interpolate a dataset to given number of time steps between existing time steps |
|
Fill missing values with a distance-weighted average. |
|
Calculate the grid box mean for all variables This is performed for each time step. |
|
Calculate the grid box max for all variables This is performed for each time step. |
|
Calculate the grid box min for all variables This is performed for each time step. |
|
Calculate the grid box sum for all variables This is performed for each time step. |
|
Calculate the grid box range for all variables This is performed for each time step. |
Masking methods¶
|
Mask a lon/lat box |
Anomaly methods¶
|
Calculate annual anomalies for each variable based on a baseline period The anomaly is derived by first calculating the climatological annual mean for the given baseline period. |
|
Calculate monthly anomalies based on a baseline period The anomaly is derived by first calculating the climatological monthly mean for the given baseline period. |
Statistical methods¶
|
Calculate the temporal mean of all variables |
|
Calculate the temporal minimum of all variables |
|
Calculate the temporal median of all variables :param over: Time periods to average over. |
|
Calculate the temporal percentile of all variables |
|
Calculate the temporal maximum of all variables |
|
Calculate the temporal sum of all variables |
|
Calculate the temporal range of all variables |
|
Calculate the temporal standard deviation of all variables |
|
Calculate the temporal cumulative sum of all variables |
|
Calculate the temporal variance of all variables |
|
Calculate the correlation correct between two variables in space This is calculated for each time step. |
|
Calculate the correlation correct in time between two variables The correlation is calculated for each grid cell, ignoring missing values. |
|
Calculate the area weighted spatial mean for all variables This is performed for each time step. |
|
Calculate the spatial minimum for all variables This is performed for each time step. |
|
Calculate the spatial maximum for all variables This is performed for each time step. |
|
Calculate the spatial sum for all variables This is performed for each time step. |
|
Calculate the spatial range for all variables This is performed for each time step. |
|
Calculate the spatial sum for all variables This is performed for each time step. |
|
Calculate the spatial range for all variables This is performed for each time step. |
|
Calculate the spatial range for all variables This is performed for each time step. |
|
Calculate the latitudinal or longitudinal centre for each year/month combination in files. This applies to each file in an ensemble. by : str Set to ‘latitude’ if you want the latitiduinal centre calculated. ‘longitude’ for longitudinal. by_area : bool If the variable is a value/m2 type variable, set to True, otherwise set to False. |
|
Calculate the zonal mean for each year/month combination in files. |
|
Calculate the zonal minimum for each year/month combination in files. |
|
Calculate the zonal maximum for each year/month combination in files. |
|
Calculate the zonal range for each year/month combination in files. |
|
Calculate the meridonial mean for each year/month combination in files. |
|
Calculate the meridonial minimum for each year/month combination in files. |
|
Calculate the meridonial maximum for each year/month combination in files. |
|
Calculate the meridonial range for each year/month combination in files. |
Merging methods¶
|
Merge a multi-file ensemble into a single file 2 methods are available. |
Splitting methods¶
|
Split the dataset Each file in the ensemble will be separated into new files based on the splitting argument. |
Output and formatting methods¶
|
Save a dataset to a named file This will only work with single file datasets. |
|
Open a dataset as an xarray object |
|
Open a dataset as a pandas data frame |
|
Zip the dataset This will compress the files within the dataset. |
|
Zip the dataset This will compress the files within the dataset. This works lazily. :param ext: New format. Must be one of “nc”, “nc1”, “nc2”, “nc4” and “nc5” . netCDF = nc1 netCDF version 2 (64-bit offset) = nc2/nc netCDF4 (HDF5) = nc4 netCDF4-classi = nc4c netCDF version 5 (64-bit data) = nc5 :type ext: str. |
Miscellaneous methods¶
|
Calculate the number of missing values |
|
Calculate the number of missing values |
|
Split the dataset into multiple evenly sized horizontal and vertical new files |
|
Collect a dataset that has been split using distribute |
|
Calculate the area of grid cells. |
|
Identify the time step when a value is first above a threshold This will do the comparison with either a number, a Dataset or a netCDF file. :param x: An int, float, single file dataset or netCDF file to use for the threshold(s). If comparing with a dataset or single file there must only be a single variable in it. The grids must be the same. :type x: int, float, DataSet or netCDF file. |
|
Identify the time step when a value is first below a threshold This will do the comparison with either a number, a Dataset or a netCDF file. :param x: An int, float, single file dataset or netCDF file to use for the threshold(s). If comparing with a dataset or single file there must only be a single variable in it. The grids must be the same. :type x: int, float, DataSet or netCDF file. |
|
Identify the final time step when a value is above a threshold This will do the comparison with either a number, a Dataset or a netCDF file. :param x: An int, float, single file dataset or netCDF file to use for the threshold(s). If comparing with a dataset or single file there must only be a single variable in it. The grids must be the same. :type x: int, float, DataSet or netCDF file. |
|
Identify the last time step when a value is below a threshold This will do the comparison with either a number, a Dataset or a netCDF file. :param x: An int, float, single file dataset or netCDF file to use for the threshold(s). If comparing with a dataset or single file there must only be a single variable in it. The grids must be the same. :type x: int, float, DataSet or netCDF file. |
|
Apply a cdo command |
|
Apply an nco command |
|
Compare all variables to a constant |
|
Method to calculate if variable in dataset is greater than that in another file or dataset This currently only works with single file datasets |
|
Method to calculate if variable in dataset is less than that in another file or dataset This currently only works with single file datasets |
|
Reduce dimensions of data This will remove any dimensions with only one value. |
|
Reduce the dataset to non-zero locations in a mask :param mask: single variable dataset or path to .nc file. The mask must have an identical grid to the dataset. :type mask: str or dataset. |
|
Set the precision in a dataset |
|
Check contents of files for common data problems. |
|
Check if files are corrupt |
A quick hack to change the grid file in North West European shelf Nemo grids. |
|
|
Set the grid type. |
|
Create a mask identifying the shallowest cell without missing values. |
|
Remove any variables, such as bnds etc., from variables. |
Ecological methods¶
|
Calculate phenologies from a dataset Each file in an ensemble must only cover a single year, and ideally have all days. |