↑ Top
Table of Contents

INTREPID task, HISTORY, report and log files (R06)

Introduction

INTREPID has a number of different types of auxiliary files connected with its processing. Here is an introduction to them:

  • .task and legacy .job files specify parameters and input and output files for INTREPID to use when it runs:
  • .task files use the current Protobuf notation and we recommend their use.
  • .job files use the legacy .job file notation that we are discontinuing.

See What are task specification files?

  • HISTORY files contain a record of INTREPID tool runs, including parameters and the paths of the input and output files for each run. The record is in either .task or legacy .job language, depending on how you execute the tool. You can use HISTORY files as:
  • A record of the tasks that the tool performed
  • A source of code to create .task files for future, similar processing tasks

See What are HISTORY files?.

  • .rpt (report) files describe the results of the tool’s operation, including any errors or other information that the user may need to have. They are part of the results of the tool’s operation.

See What are Report files?.

  • .log files contain a running commentary that the tool produces as it runs the process. They are useful for checking correct operation or locating problems.

See What are Log files?.

What are task specification files?

Task specification files are text files containing a number of parameters required to execute one or many INTREPID tasks. These filenames always end in .task, so they are also referred to as task files.

We have recently introduced a new auxiliary file notation based the Protocol Buffers (Protobuf) structure. This chapter describes the use of both the Protobuf system and the legacy system.

Task files using Protobuf have the extension .task.

Task files using the legacy INTREPID ‘Parms Block Structured’ language notation, which, in the future, we will no longer support, have extension .job.

We are working to adopt the Protobuf system universally, but some areas still require the legacy notation for some functions. For more information, see Loading and saving .task files.

This chapter describes common properties of task specification files, including some of the more advanced features available through the INTREPID batch language.

Here is an example of a task specification file (.task).

IntrepidTask { ProjConv { XIN: "./ebagoola_S..DIR/x"; YIN: "./ebagoola_S..DIR/y"; XOUT: "./ebagoola_S..DIR/x_GDA94"; YOUT: "./ebagoola_S..DIR/y_GDA94"; OutputProjection { map_projection: "MGA54"; spatial_datum: "GDA94";} UpdateSurveyInfo: true; } }

The IntrepidTask { – } text block contains a complete description of files and parameters required to run an INTREPID task. The ToolName parameter specifies the filename of the INTREPID executable. In this example it projconv.exe, the executable binary for the Projection Conversion Tool.

Here is the same example in the legacy notation (.job).

Process Begin Name = projconv XIN = ../datasets/ebagoola_S..DIR/x YIN = ../datasets/ebagoola_S..DIR/y XOUT = ../datasets/ebagoola_S..DIR/x_GDA94 YOUT = ../datasets/ebagoola_S..DIR/y_GDA94 Parameters Begin ToDatum = "GDA94" ToProj = "MGA54" UpdateSurveyInfo = Yes Parameters End Process End

About the Protobuf-based task file language

The Protobuf-based .task file language provides capabilities that were not available with the previous Parms .job file system:

  • The language is defined formally using Protobuf *.proto files. We provide these as part of the distribution in the api directory, so you can easily see which keywords are available for which process and the values they can take.
  • The technology provides better error detection, so if there is an error in a *.task file, the problem can be identified more easily.
  • For advanced users there is high-level tooling support that makes integration easier.

Converting and editing task specification files

Converting

The Project Manager tool has a legacy .job to .task file converter. See “Converting legacy Parms (.job) files to Protobuf (.task) files” in INTREPID Project Manager (T02).

Editing

For both interactive and batch use of INTREPID tools, it is useful to be able to edit a task specification file, using a text editor, to:

  • Adjust file and parameter settings for a task, or
  • Create a new task specification file for a different task, using some of the text of the original file as a template.

Creating new task files

You can create new task specification files by:

Task file examples supplied with INTREPID

Location of sample .task and .job files and the data for them

With INTREPID we provide collections of sample .task files and corresponding sample databases for them to process.

For the time being, we also currently provide collections of legacy .job files.

Where install_path is the path of your INTREPID installation, the folder containing these collections is
install_path\sample_data\examples\task
and (for a limited time)
install_path\sample_data\examples\jobs_old

The sample datasets that these .task and .job files access and process are in this folder:
install_path\sample_data\datasets

For example, if INTREPID is installed in
C:\Intrepid\Intrepid6.0.1123a1078bde_x64,
then you can find the sample datasets in
C:\Intrepid\Intrepid6.0.1123a1078bde_x64\sample_data\datasets

More information

For more information about installing sample data, see “Sample datasets – installing, locating, naming” in INTREPID Guided Tours introduction (G01)

For a more detailed description of INTREPID datasets, see Introduction to the INTREPID database (G20). For even more detail, see INTREPID database, file and data structures (R05).

Using task specification files

See “How to start INTREPID – Overview” in Introduction to INTREPID (R02) for complete instructions. This section describes differences between .task and legacy .job files in the different modes of running INTREPID tools.

The following table contains a summary of the current possible ways to use task specification files and their availability with .task and legacy.job files.

Mode

Tool executed

Operation

.task

.job
(legacy)

Interactive

Tool doing the work

Load task file to set parameters

Works for some tools, with more planned

yes

Save task file to remember parameters for reuse or creating a batch file

Command line

Open a tool for interactive use and load parameters
(using –input parameter)

works for some tools

Execute single tool with task file (using –batch parameter)

yes

yes

Batch Process Manager – see “Using task specification files” in INTREPID Batch Process Manager (T01)

Execute multiple tool batch task (using –batch parameter)

yes

yes

Using task specification files in interactive mode

If you are using an INTREPID tool for the first time you will most likely use the interactive interface to specify all the required files and parameters.

Most INTREPID tools also allow you to:

  • Load a task specification file to pre-configure an interactive session with the tool with all the required files and parameters it needs to execute the task.
  • Save a task specification file to save the current file and parameter settings. This is convenient if you may wish to repeat the process later. You can also edit the file to specify different datasets or parameters.

Loading and saving .task files

With the tools that have fully changed over to interactive .task file operations, you can:

  • Load .task files for configuring an interactive session.
  • Save .task files if you want to repeat an interactive or batch operation with:
  • The same settings or
  • Settings that you later specify by editing the .task file.

Most tools are now able to load .task files to configure their settings for interactive use.

Interactive .task file saving is currently available in the following tools:

Note: INTREPID may ignore settings and parameters that are available in the .task file Protobuf definition but not shown in the tool window. If you want to use settings that are only available in the .task file, then run the tool in batch mode.

Loading and saving legacy .job files

For the INTREPID tools not listed in Loading and saving .task files, for the time being, you can:

  • Load legacy .job files for configuring an interactive session.
  • Save legacy .job files for repeating an interactive or batch operation with:
  • The same settings or
  • Settings that you later specify by editing the legacy .job file.

Using task specification files in command line mode

Task specification files consist of a sequence of INTREPID task language statements. This means that they can be executed not only interactively but also as batch processes, independently of the interactive tools. In fact, the batch language provides options that are not available through the interactive tools. You can create long and complex task files which can only be run as batch processes.

Running task specification files from a command line

You can use a command to run batch tasks specified by task files. It works with both .task and legacy .job files

Type the command followed by the switch -batch, followed by the name of the task specification file. If the task file involves only one tool, for example, a Gridding task, then use the file name of the executable program. For example:

jGridding.exe -batch eba1.task

However if the task file contains calls to more than one application, or if it contains some of the more advanced features of the batch language, then you must use the Batch Process Manager (see INTREPID Batch Process Manager (T01)). The command must start with fmanager.exe. Example:

fmanager.exe -batch eba1.task

For this reason we recommend (and it is just as easy) that you always use the command fmanager.exe to execute an INTREPID task file in batch mode.

Note: On Windows platform the .exe is not required.

Opening a tool in interactive mode from the command line

You can use a command to launch a tool in interactive mode and load the parameters specified in a task file. Use the switch -input with the command. Example:

marinelevel.exe -input loop_A_vs_B.task

You may want to use this feature if:

  • You have a task with several steps, so that you can pause the task to carry out an interactive operation.
  • You want to create a shortcut for a frequently performed operation on a dataset

Using the command in this way is the same as launching the tool from the Project Manager and using Open Taskfile....

Note: For certain INTREPID tools this type of command currently works only with legacy .job files. For a list of the tools that can use .task files, see Loading and saving .task files or contact our technical support service.

What are HISTORY files?

Whenever you use INTREPID, it records your activities in a history file. This is a text file with the name HISTORY. It can act as an audit trail for your processing. HISTORY consists of a series of task specification files. You can extract sections from it to use in task specification files or even use a whole history file for a batch task.

Note that in the current release of INTREPID, the history file contains task specifications that may be in either in the Protobuf .task or legacy .job language.

The following list shows which format INTREPID will use

Mode

Circumstances

Format

Batch

command used a .task file

.task format

Batch

command used a legacy .job file

legacy .job format

Interactive

Tools listed in Loading and saving .task files

.task format

Interactive

Tools not listed in Loading and saving .task files

legacy .job format

Using HISTORY to reprocess a dataset

You can specify HISTORY as the task specification file for a batch mode Batch Process Manager task. In doing this you can use the HISTORY file’s audit trail to ’replay’ an entire set of processes on a dataset.

Depending on processes that you want to carry out and the contents of your HISTORY file, make sure you attend to the following:

  • Ensure that you do not overwrite the raw data (for example, make sure that you create new vector dataset fields for intermediate results as you proceed on the first occasion).
  • Avoid direct editing using the Spreadsheet tool. If you need to do this, ensure that you have good records. The current version of INTREPID does not record Spreadsheet tool operations in HISTORY.
  • Only use standard INTREPID tools for the process. If you do this, all of the steps of the process will be recorded in HISTORY.
  • If you want to make some changes to the process before the batch mode operation, you can, of course, edit HISTORY using a text editor. If you do this, then:
  • You can extract sections of it and save them as task specification (.task or legacy .job, depending on the language that they use) files if you do not wish to repeat all steps;
  • Be careful not to accidentally corrupt HISTORY. Edit it with a text editor, not a word processor.
  • If you want to build a new HISTORY file starting with a certain processing step, simply delete the existing HISTORY before you commence this process. INTREPID will create a new HISTORY starting with the current process.
  • Note that any direct editing of HISTORY compromises the concept of HISTORY being an audit trail.

See also:

Creating task specification files

Creating task specification files in an interactive session

You can create a .task task specification file using Save Taskfile or Save Options from the File menu in most INTREPID tools. For a list of the tools that now create .task files, see Loading and saving .task files.

To create a task specification file with an INTREPID tool
  1. Specify all files and parameters.
  2. If possible, execute the task (choose Apply) to ensure that it will work.
  3. Choose Save Taskfile or Save Options from the File menu. Specify a task specification file INTREPID adds the extension .task. See Loading and saving .task files for a list of tools that now use .task files interactively. INTREPID creates the file with the settings current at the time of the Save Taskfile or Save Options operation.

Creating task specification files with a text editor

If you know the correct syntax you can create a task specification file using a text editor, either from scratch or by editing an existing task specification file.

Ensure that it has the extension .task or legacy .job (depending on the language) when you save it.

Extracting task specification files from HISTORY

If you have recently executed a task similar to the one for which you are creating the task specification file, you can use a text editor to extract the text for the task from HISTORY, then edit it to fit your new requirements. Save it with the extension.task if it is in Protobuf or .job if it is in legacy Begin – End format.

See Using HISTORY to reprocess a dataset.

Task specification file syntax

The current INTREPID task files syntax (.task files) is based on Google Protobuf messages. To view the syntax definitions, examine the *.proto files in your {install_path}/extras/protobufs directory.

The legacy .job task specification files are text files with the standard INTREPID Begin – End block syntax (See “INTREPID Auxiliary files” in INTREPID database, file and data structures (R05) for a description of the legacy .job syntax).

For a discussion and example of task specification file format for a particular tool, see the section usually entitled ‘Displaying options and using task specification files’ usually placed at the end of the chapter describing the tool.

Because new features are regularly added to INTREPID, the task specification file language may change. To make sure you are using the most up-to-date syntax, run the INTREPID tool you require interactively and save a new task specification file or examine HISTORY. (Note that in the current release of INTREPID, this does not apply. The HISTORY file contains task specifications in the previous legacy .job language. Please contact our technical support service for assistance with this.)

Compatibility with earlier INTREPID versions

In some cases, changes to the task specification file language may prevent INTREPID from successfully executing older task specification files. We do not guarantee backwards compatibility for existing task specification files when we release a new version of INTREPID. We recommend that you check your existing task specification files against entries in HISTORY or new task files created interactively. Do not rely upon older task files to execute properly in later versions of INTREPID.

Case sensitivity

Task specification file keywords are case sensitive.

Compatibility between Windows and Linux

If you are executing a task specification file created under the other operating system, please note the following:

  • INTREPID automatically understands / and \\ characters as required in path specifications. Note that if you are using backslashes \\ in a path, they need to be double.
  • INTREPID .task files may contain full or relative paths but not system parameters, such as environment variables.
Full path examples:

Full Path

Path

Linux full path:

/disk1/data/Projects/gridarea

Windows full path:

d:\\data\\Projects\\gridarea

Relative paths and the working directory

When you create a task specification (.task – see Loading and saving .task files) using Save Taskfile or Save Options from the File menu in a tool, or extract a .task or legacy .job file from HISTORY, INTREPID inserts full directory paths for file and dataset specifications.

You can edit the full directory paths and change them to relative paths. If you do this, INTREPID will use the file and directory references relative to the working directory at the time you execute the task specification file.

Note: If there are relative paths in a task specification file you must ensure that you set the correct working directory before executing the task.

If you move a task specification file with relative paths you must do one of the following:

  • Move it as part of the project with the associated directory structure so that the relative paths can remain unchanged;
  • Edit it to modify the relative paths;
  • Edit it to include full paths.
  • Edit it to use a variable to define the path (.task files only).

For example, use a system parameter or environment variable MYDIR to contain the path and use $(MYDIR) to refer to it in the task specification files. A typical statement could be (in each language, respectively)

ZOUT : "$(MYDIR)/ebagoola_ST/mlevel_mag1";

ZOUT = $(MYDIR)/ebagoola_ST/mlevel_mag1

See Command line replaceable parameters below for instructions about using variables.

Relative and full path examples

.task

Example of statement with relative path

ZOUT : "ebagoola_ST/mlevel_mag1";

Example of statement with full path

ZOUT : "F:/DATA/disk1/surveys/ebagoola_ST/mlevel_mag1";

Legacy .job

Example of statement with relative path

ZOUT = ebagoola_ST/mlevel_mag1

Example of statement with full path

ZOUT = F:/DATA/disk1/surveys/ebagoola_ST/mlevel_mag1

Batch Process Manager task operations

This section describes operations that are only available in batch mode. To perform these operations we need to (using eba1.task as an example):

  • Use the interactive Project Manager to run the task file eba1.task (it launches Batch Process Manager) OR
  • Use the Batch Process Manager command ‘fmanager -batch eba1.task

These actions invoke the Batch Process Manager to execute the batch language statements contained within the task file eba1.task. (The same goes for legacy .job files)

The Batch Process Manager has some special batch mode capabilities which are not available interactively. These features are as follows:

Your INTREPID installation includes examples of task specification files showing the correct syntax. For details, see Task file examples supplied with INTREPID.

You can use the HISTORY file to produce a task specification file for the Batch Process Manager. See Using HISTORY to reprocess a dataset

Support specifications for all INTREPID tools

You can execute any batch task with Batch Process Manager or a single tool:

  • From the command line or
  • Through Project Manager (which launches Batch Process Manager).

See “How to start INTREPID – Batch mode” in Introduction to INTREPID (R02) for instructions.

Execute multiple tasks sequentially

The Batch Process Manager can execute a number of task specifications placed one after another in a task specification file. Here is an example, showing a decorrugation process followed by a microlevelling process.

Example (.task)

IntrepidTask { Decorrugate { ToolName : "decor"; InputGrid : "smooth_grid_t.ers"; OutputGrid : "corr_grid.ers"; ... } } IntrepidTask { MicroLevel { ToolName : "mlevel"; InputGrid : "corr_grid.ers"; ZIN : "ebagoola_ST..DIR/smooth_mag"; LineType : "ebagoola_ST..DIR/linetype"; ZOUT : "ebagoola_ST..DIR/mlevel_mag1"; ... } }

Example (legacy .job)

Process Begin Name = decorrugate InputGrid = smooth_grid_t.ers OutputGrid = corr_grid.ers Parameters Begin ... Parameters End Process End Process Begin Name = microLevel InputGrid = corr_grid.ers ZIN = ebagoola_ST..DIR/smooth_mag LineType = ebagoola_ST..DIR/linetype ZOUT = ebagoola_ST..DIR/mlevel_mag1 Parameters Begin ... Parameters End Process End

Fixed parameter substitution within the task file

You can declare parameters and assign values to them within a task specification file. Normally these parameters appear at the beginning of the file. You can then use the parameters to assign values in the file. The assigned values can be numeric or character strings. You place a $ before the name of a parameter when you use it to supply a value. This is mainly useful when you have a number of tasks to perform in the same task file. In the following example a fixed parameter called Corrections is declared and given a value of corr_grid1.

Example (.task)

Repeat { Corrections : "corr_grid1.ers"; Intrepid_Task { Decorrugate { Tool_Name : "decorrugate"; InputGrid : "smooth_grid_t.ers"; OutputGrid : "$Corrections"; ... } } Intrepid_Task { MicroLevel { Tool_Name : "mlevel"; InputGrid : "$Corrections"; ZIN : "ebagoola_ST..DIR/smooth_mag"; LineType : "ebagoola_ST..DIR/linetype"; ZOUT : "ebagoola_ST..DIR/mlevel_mag1"; ... } } }

Example (legacy .job)

Corrections = corr_grid1.ers Process Begin Name = decorrugate InputGrid = smooth_grid_t.ers OutputGrid = $Corrections Parameters Begin ... Parameters End Process End Process Begin Name = microLevel InputGrid = $Corrections ZIN = ebagoola_ST..DIR/smooth_mag LineType = ebagoola_ST..DIR/linetype ZOUT = ebagoola_ST..DIR/mlevel_mag1 Parameters Begin ... Parameters End Process End

Variable parameter substitution within the task file

Variable parameters can also be declared. These are used in combination with a Repeat loop (see below). For each repeat cycle the parameter takes a different value, assigned from a list enclosed within braces. Parameter values in the list are separated by one or more spaces.

You must follow a variable name reference with a character that is not a letter or number.

For example, if you are referencing the variable $MYDIR, You can use the notation TempDir : "$MYDIR/temp"; but not TempDir : "$MYDIRtemp";

The following example shows a combination of fixed and variable parameters declared at the start of the task file, and used to provide values within the Process block.

A variable name can also be constructed from other declared variable names. In the example, ‘OutputImage’ is created using a combination of declared variables and a text string. If the underscores are omitted the strings will be concatenated.

Example (.task)

Repeat { dataset : "Ebagoola..DIR"; proj : "mga53"; band [lev_mag v_deriv]; element : [TMI 1VD]; IntrepidTask { Gridding { ToolName : "newgridding"; XIN : "$dataset/$(proj)E"; YIN : "$dataset/$(proj)N"; ZIN : "$dataset/$band"; OutputImage : "$(dataset)_$(element)_gda94_$(proj)"; ... } } }

Example (legacy .job)

Repeat Begin dataset = Ebagoola..DIR proj = mga53 band={lev_mag v_deriv} element={TMI 1VD} Process Begin Name = newgridding XIN = $dataset/$(proj)"E" YIN = $dataset/$(proj)"N" ZIN = $dataset/$band OutputImage = $(dataset)_$(element)_gda94_$(proj) Parameters Begin ... Parameters End Process End Repeat End

Repeat one or more tasks a number of times using variable parameters

The Batch Process Manager can repeat a batch task, using different parameters each time according to a list of parameter values you provide. INTREPID determines the number of repeat cycles by the maximum number of entries in any parameter list. If there are several lists, INTREPID uses the first value in each list for the first cycle, then the second items for the second cycle, etc..

In the following example, INTREPID will import three files dat1, dat2 and dat3 into three separate vector datasets eba_1, eba_2 and eba_3.

Example (.task)

Repeat { SurveyPart : [dat1 dat2 dat3]; OutData : [eba_1 eba_2 eba_3]; IntrepidTask { Import { ToolName : "import"; Input : "/disk1/import/i55data/$SurveyPart"; Output : "/disk1/surveys/$Outdata"; Format : "AsciiColumns"; AsciiColumns { FixedLength : false; SkipRecords : 0; DDF : "/disk1/import/gravity_database.ddf"; NoError : false; ReportDiagnostics : false; } } } }

Example (legacy .job)

Repeat Begin SurveyPart = {dat1 dat2 dat3} OutData = {eba_1 eba_2 eba_3} Process Begin Name = Import Input = /disk1/import/i55data/$SurveyPart Output = /disk1/surveys/$Outdata Parameters Begin Format = AsciiColumns AsciiColumns Begin FixedLength = No SkipRecords = 0 DDF = "/disk1/import/gravity_database.ddf" NoError = No ReportDiagnostics = No AsciiColumns End Parameters End Process End Repeat End

Note: It is also possible to use nested Repeat loops.

Command line replaceable parameters

When you launch a task from a command line, you can add parameters to the command line (separated by spaces) which correspond to parameter number references in the body of the task specification file. For example the third, fourth and fifth parameters on the command line correspond to the notations $3, $4, $5 in the task specification file. These notations are called replaceable parameters.

INTREPID will substitute values of replaceable parameters from the command line. Note that switches (e.g., –batch) count as parameters in the numbering system.

For example, command lines could be:

(Batch mode)

gridding.exe -batch eba1.task raw_mag raw_grid

fmanager.exe -batch eba1.task raw_mag raw_grid

(Interactive mode)

jGridding.exe -input eba1.task raw_mag raw_grid

The task specification file (eba1.task) appears below.

Gridding will use

  • The third parameter raw_mag as the value for the file name of ZIN ($3) and
  • The fourth parameter raw_grid as the value for the file name of OutputImage ($4).

Example (.task)

Intrepid_Task { Tool_Name : "Gridding"; Parameters { ... } XIN : "ebagoola_S..DIR/x"; YIN : "ebagoola_S..DIR/y"; ZIN : "ebagoola_S..DIR/$3"; LineType : "ebagoola_S..DIR/linetype"; OutputImage : "$4"; }

Example (legacy .job)

Process Begin Name = Gridding Parameters Begin ... Parameters End XIN = ebagoola_S..DIR/x YIN = ebagoola_S..DIR/y ZIN = ebagoola_S..DIR/$3 LineType = ebagoola_S..DIR/linetype OutputImage = $4 Process End

System parameter references

INTREPID can insert values of system parameters and environment variables in task specification file statements. Use the notation $system_parameter to specify a system parameter value (where system_parameter is the name of a system parameter). If you are appending some text to the value of the system parameter, put its name in parentheses

Example: If you have the following statement:

.task

TempDir : "$(INTREPID)/temp";

Legacy .job

TempDir=$(INTREPID)/temp

INTREPID will recognise $INTREPID as a system parameter, look up its value and use it for the string to be combined with /temp and assigned to TempDir.

For full instructions on using system parameters and environment variables, see “About INTREPID system parameters” in INTREPID system parameters and install.cfg (R07).

References to parameter values in auxiliary files

INTREPID will look up values assigned to keywords in auxiliary files for use in task specification files. Use the following notation in your task specification file:

$[auxfile]blockname.keyword or

$[auxfile]blockname.sub_blockname.keyword

auxfile is the INTREPID auxiliary file

blockname is the name of a begin – end block

sub_blockname is the name of a begin – end block within the blockname block

keyword is the name of a keyword within the begin – end block.

Example

The file ebagoola352.ers includes the following lines:

DatasetHeader Begin ... CoordinateSpace Begin Projection = "TMAMG54" CoordinateType = EN Units = "METERS" Datum = "AGD66" CoordinateSpace End ... DatasetHeader End

If you have the following line in your task specification file:

For .task

Projection : $[ebagoola352.ers]DatasetHeader.CoordinateSpace.Projection";

For legacy .job

Projection=$[ebagoola352.ers]DatasetHeader.CoordinateSpace.Projection

INTREPID will look up the auxiliary file and insert the value TMAMG54 for the parameter Projection.

Specific Action commands – Echo and Command

Specific Action commands can be used for specific purposes.

In a large complex task specification file it may be necessary to output a text string into the Log file for easier monitoring of task progress. This can be achieved using the Echo Action command. (For more about Log files, see What are Log files?)

Example (.task)

IntrepidTask { FileManager { ToolName : "fmanager"; Action : "Echo"; Text : "And tomorrow will be like today, only more so."; } }

Example (.job)

Process Begin Name = filemanager Parameters Begin Action = Echo Text = And tomorrow will be like today, only more so. Parameters End Process End

The Command Action command allows access to Operating System commands from within a task specification file.

Example (.task)

IntrepidTask { FileManager { ToolName : "fmanager"; Action : "Command"; Script : "ls"; Args : "-al"; Pause : True; } }

Example (legacy .job)

Process Begin Name = filemanager Parameters Begin Action = Command Script = ls Args = "-al" Pause = yes Parameters End Process End

The ‘Args’ parameter can even be a shell script. The following example invokes a Bourne shell which runs a script called ‘importscript’. This script performs a complicated extraction and formatting of data and then calls another Intrepid task file which import the reformatted data to an Intrepid dataset. Control is then returned to the following task file.

Example (.task)

IntrepidTask { FileManager { ToolName : "fmanager"; Action : "Command"; Script : "sh -c"; Args : "./importscript"; Pause : False; } }

Example (legacy .job)

Process Begin Name = filemanager Parameters Begin Action = Command Script = sh -c Args = "./importscript" Pause = no Parameters End Process End

The Spawn Action command enables a separate task specification file to be launched and executed midway through execution of the parent task.

Example (.task)

IntrepidTask { filemanager { Action: Spawn; Job: echo.task; } }

What are Report files?

Many INTREPID tools automatically produce report files when they run. Report files are text files with extension.rpt. They describe the intermediate values (if required) and results of the process as well as reporting any errors.

For more information about report files, see the individual chapter about the tool you are using.

What are Log files?

INTREPID tools write logging information as they perform their tasks. This information can be helpful to diagnose any issues during processing. Project Manager can display this information to you both while a tool is running and afterwards. Logging information is available for a selected dataset, a selected task specification file, or a selected directory.

By default, INTREPID stores logs in the following directory, where <user> is an example of a user’s login name and C: is an example of the man drive of the computer.

C:/Users/<user>/AppData/Roaming/Intrepid Geophysics/Logs

jpmlogfl.png

Warning: When the default path is used for .log files they are kept indefinitely. Depending on the spare space on your hard-drive it may be good practice to delete old .logs or place the logs directory under Windows/temp

INTREPID tools log information with some information describing each log entry. All log entries have a timestamp, an application name, a level, a PID, and a component, along with the textual log entry itself.

  • The timestamp is the date and time at which the log entry was created.
  • The application name is the display name for the tool.
  • The level is one of "debug", "note", "info", "warn", and "error", and describes how severe the process considers the situation causing the log entry to be created.
  • The PID (Process ID) identifies the instance of the tool. For example, if you ran two instances of jGridding simultaneously, they would each have a different PID. PIDs are unique only for a given time on a particular computer.
  • The component is a tag indicating which subsystem created the log entry. A single process will use several components.

Changing destination of .log files.

The write destination of .log files can be changed to any other writeable (not restricted) file location by changing the system/user variable INTREPIDLOG.

On Windows systems, search for environment variables, open Edit your system environment variables and under the Advanced Tab select Environment Variables...

Under User Variables Choose to add a New..

Set the Variable name to INTREPIDLOG

Set the Variable value to <Desired_Directory>\batch.log.

Specify the <Desired_Directory> by using Browse Directory... After the directory is selected add \batch.log to the end of the path. Press OK.

INTREPID will now write log files to this new file.

Warning: The directory MUST exist in order for .log files to be written.

Tip: All logs files that previously would be separate files, will be contained in this newly set file.

env_variables.pngenv_variables010.png

Adding text

You can instruct INTREPID to write your own extra text into the Log file using the Echo Action command. See Specific Action commands – Echo and Command

Viewing log entries

Project Manager has a dedicated log viewer that enables you to view log entries, filter the display according to the level and by text found in the log entry, and to export the entire log to a text file. For more information, see “Displaying diagnostic information from Log files” in INTREPID Project Manager (T02).

Tailing a task (watching progress of a task) using the log file

You can ‘tail’ or watch the progress of a task using log files.

Linux

In Linux use the tail command.

Windows

In Windows there are several methods. We show here how to do it with the free software Notepad++ from https://notepad-plus-plus.org/. Firstly, set the preferences in Notepad++ as follows:

  1. In Notepad++, Choose Settings > Preferences > MISC. Check the following:
    • Update silently
    • Scroll to the last line after update
    tasknppr.png
    tasknpup.png
  2. Choose Close.

Tail the task

  1. Start the task
  2. Using the date directory, application name and date and time of the file, locate the .log file of the task.
  3. (Windows) Open the file in an automatically updating text viewer such as Notepad++
    (Linux) Tail the file.