Have you wanted to log to a QVD in a space within your Qlik SaaS tenant? I’m sure there are multiple ways to do this, but this is what I’ve come up with.

High Level Overview

  1. Log to a variable with pipe field delimiters and line break row delimiters
  2. Create a temporary Qlik Sense app
  3. Set the load script to the contain an Inline table using the Log variable
  4. Reload the app
  5. Delete the app
  6. *Optional: Log the data to the output using a markdown table

Automation

TL;DR

Don’t care about the details and just want the code? You can find it here.

Instructions on how to upload an automation: Help | Knowledge Article

Breaking It Down

Block

Details

To keep things clear across the different automations, I typically create a variable which contains the guid of the automation.

You can easily create a URL to in a Qlik Sense App using https://<tenant>.<region>.qlikcloud.com/automations/{automationId}

Settings

Block

Details

In most cases you will have actual things going on in your automation. However, for the sake of this article I am just using a test value and loop for my logging input.

[
    {
        "id": 1,
        "message": "success"
    },
    {
        "id": 2,
        "message": "warning"
    },
    {
        "id": 3,
        "message": "error"
    }
]

Settings

Block

Details

Here we will be looping over the test value we are using for our example. To do so we are using the Object formula to convert the string to an actual Object.

Settings

Block

Details

In this block what I am doing is appending to the log (string) variable. Depending on your data you will be logging, the functions you might need will be different. However, what I am doing is converting an object to a CSV string, using a pipe (|) delimiter.

Convention

  • Pipe – Field Delimiter
  • Line Break – Row Delimiter

In our case the Append operation automatically performs a line break before adding data. However you may need to use the LineBreak formula depending on how you are logging you information.

This will format the data for our Inline Load later in another block. The goal is to be able to have the load script statement result in something like so:

example:
LOAD * INLINE ` first_field|second_field
1|One
2|Two`(delimiter is '|');

We are using ticks instead of brackets for the Inline statement since brackets are very common in JSON and can become an easy error depending on the data you are working with.

Settings

Block

Details

Optionally, you can perform some nice looking logging to the automation output using markdown. In this block, I am using the variable to dynamically generate a markdown table. Another reason why I use the pipe delimiter. While you can change the delimiter for the Inline Load, you cannot change the markdown delimiter.

Settings

Output

Block

Details

Here we create a temporary application. At this point in time there is no Session App block and working with websockets, if at all possible, in the Custom Code block would make this extra complicated. So we will be creating is a pseudo-ephemeral app by creating an app and deleting it later.

I prefer to use the Automation Id in the name, that way I can identify apps created but not deleted during automation failures. It also helps to add a description, giving confidence to your coworkers to delete your clutter. At this point the description is mandatory, so might as well…

Lastly, I chose to leave the Space Id empty, which will create it in my Personal space. In most cases I have this set to a particular space, but I kept it simple so you don’t need to update this setting in your version.

Settings

Block

Details

Here is a piece you will definitely need to customize to your needs. This will define your log structure and where your log will be stored to.

Script

// Config
Let vAutomationId = '{$.automationId}';
Let vLog = 'lib://Automations:DataFiles/log.qvd';
Let vLogTime = TimeStamp(ConvertToLocalTime(Now(),'EST'));


// Log Schema
log:
Load 
    Null()              as automation_id,
    Timestamp(Null())   as log_time,
    Null()              as id,
    Null()              as message
AutoGenerate
    (0);


// If log exists, load the history
If FileSize('$(vLog)') > 0 Then

    Concatenate(log) Load * From [$(vLog)](qvd);

End If;

// Log Data
Concatenate(log)
Load 
    '$(vLogTime)'       as log_time,
    '$(vAutomationId)'  as automation_id,
    *
Inline `
id|message
{$.log}
`(delimiter is '|');


// Export Log
Store log into [$(vLog)](qvd);

Settings

Block

Details

Even though we have updated the load script, the actual changes aren’t applied until we save the app. This step solidifies our changes, updating the app before it gets reloaded.

Settings

Block

Details

Trigger the application reload so that it actually performs the logging we set in its script. At this stage the QVD gets created.

Settings

Block

Details

This step is to perform the garbage collection of the temporary app, making it actually temporary and not just clutter.

Settings

Block

Details

In most scenarios if you are logging to a QVD, you have an application which loads that data. This last step triggers that application’s reload.

I personally don’t want to wait for that application to finish reloading before I have my automation finish, so I have the Run Mode to “Start reload and continue”. This way the reload time isn’t a part of my automation time.

Settings

Output

So wrapping up, this will create a QVD which contains your logging data of your automation, which can then be used by a Qlik Sense app to analyze.