Examples¶

Overview¶

Infero comes with examples that demonstrate how to use Infero API’s for different languages. Examples can be found in:

<infero-source-path>/examples

And when compiled, the corresponding executables are found in:

<infero-build-path>/bin

The examples show how to use Infero for a multi-input single-output Machine Learning model from C, C++ and Fortran (same model and input data are used for all the cases, so also same output is expected).

1_example_mimo_c.c

2_example_mimo_cpp.cc

3_example_mimo_fortran.F90

4_example_mimo_thread.cc

Run the examples¶

The examples can be run as follows (in this specific case shown below, the onnx backend is used - therefore to run the example as below, make sure that ONNX backend is installed - see Build & Install).

Note that here below <path/to/mimo/model> is: <path/to/infero/sources>/tests/data/mimo_model

C example:

cd <path/to/infero/build>
./bin/1_example_mimo_c <path/to/mimo/model>/mimo_model.onnx onnx input_1 input_2 dense_6

C++ example:

cd <path/to/infero/build>
./bin/2_example_mimo_cpp <path/to/mimo/model>/mimo_model.onnx onnx input_1 input_2 dense_6

Fortran example:

cd <path/to/infero/build>
./bin/3_example_mimo_fortran <path/to/mimo/model>/mimo_model.onnx onnx input_1 input_2 dense_6

C++ threaded example:

cd <path/to/infero/build>
./bin/4_example_mimo_thread <path/to/mimo/model>/mimo_model.onnx onnx input_1 input_2 dense_6

Code Explained¶

The examples are extensively commented to describe the usage of the API’s step-by-step. A brief description of the main sections from the Fortran example is also reported here below (for the full example, refer to 3_example_mimo_fortran.F90).

This section below contains the declaration of the necessary input variables. t1 and t2 are the fortran arrays containing input data and t1_name and t2_name are the names of the input layers to which the tensors will be assigned.

! input tensors
real(c_float) :: t1(n_batch,32) = 0
real(c_float) :: t2(n_batch,128) = 0

! names of input layers
character(len=128) :: t1_name
character(len=128) :: t2_name

The association between tensors and names of the corresponding input layers is then made through a key/value container of type fckit_map (here below the necessary declarations):

! auxiliary fckit tensor wrappers
type(fckit_tensor_real32) :: tensor1
type(fckit_tensor_real32) :: tensor2

! key/value map for name->tensor
type(fckit_map) :: imap

Output tensor(s) are declared and arranged into an fckit_map in the same way.

! output tensor
real(c_float) :: t3(n_batch,1) = 0

! name of output layer
character(len=128) :: t3_name

! auxiliary fckit tensor wrappers
type(fckit_tensor_real32) :: tensor3

! key/value map for name->tensor
type(fckit_map) :: omap

The type for the machine learning model is called infero_model:

! the infero model
type(infero_model) :: model

Input tensors are filled row-wise with dummy values for this example and the fckit_map is filled in:

! fill-in the input tensors
! Note: dummy values for this example!
t1(1,:) = 0.1
t1(2,:) = 0.2
t1(3,:) = 0.3

t2(1,:) = 33.0
t2(2,:) = 66.0
t2(3,:) = 99.0

! init infero library
call infero_check(infero_initialise())

! wrap input tensors into fckit_tensors
tensor1 = fckit_tensor_real32(t1)
tensor2 = fckit_tensor_real32(t2)

! construct the fckit input map
imap = fckit_map()

! insert entries name+tensor into the input map
call imap%insert(TRIM(t1_name), tensor1%c_ptr())
call imap%insert(TRIM(t2_name), tensor2%c_ptr())

Same thing is done for the output tensor

! wrap output tensor into fckit_tensor
tensor3 = fckit_tensor_real32(t3)

! construct the fckit output map
omap = fckit_map()

! insert entry name+tensor into the output map
call omap%insert(TRIM(t3_name), tensor3%c_ptr())

Configure and call infero inference method

! YAML configuration string string
yaml_config = "---"//NEW_LINE('A') &
  //"  path: "//TRIM(model_path)//NEW_LINE('A') &
  //"  type: "//TRIM(model_type)//c_null_char

! get a inference model model
call infero_check(model%initialise_from_yaml_string(yaml_config))

! run inference
call infero_check(model%infer(imap, omap))

Print inference statistics, configuration and output values

! explicitely request to print stats and config
call infero_check(model%print_statistics())
call infero_check(model%print_config())

! print output
call infero_check(oset%print())

Finally free the allocated memory for the input and output tensor sets and, free the model and finalise the library itself

! free the model
call infero_check(model%free())

! finalise fckit objects
call tensor1%final()
call tensor2%final()
call tensor3%final()
call imap%final()
call omap%final()

! finalise library
call infero_check(infero_finalise())