assert
Draft in Progress
This document is a draft and is under development.
Description
- Declare a condition that should be true.
- Throw an error if the test breaks for an unexpected reason.
- End in failure if the assertion proves false.
Attributes
List of assert()
Attributes
assert()
for Database
- >> REF:
assertion-mode: - >>
POSSIBLE VALUES
:
equals - - default mode
empty
notEmpty
failure-id: - >> TYPE: string
assert()
for File
- >> REF:
source-file: - >> TYPE: string - - REQUIRED
sql: - >> TYPE: string
sql-script: - >> TYPE: string
target: - >> TYPE: string
classifier: - >> TYPE: string
context: - >> TYPE: string
producer: - >> TYPE: string
failure-id: - >> TYPE: string
assertion-mode: - >>
POSSIBLE VALUES:
equals - - default mode
exists
notExists
empty
notEmpty
reference-file-type: - >> REF:
base-source-reference-file-type: - >> REF:
source-reference-file-type: - >> REF:
target-reference-file-type: - >> REF:
column-list: - >> TYPE: unique list of strings
column-list-mode: - >>
POSSIBLE VALUES:
include
exclude
assert()
for Data Sets
data-set-name: - >> TYPE: string - - REQUIRED
data-set-id: - >> TYPE: string
failure-id: - >> TYPE: string
ignore-data-set-properties: - >> TYPE: boolean
Individual
assert()
Attributes
assert()
for Database Context
[Database Feature Module - Configuration properties not listed here.]
connection-id:
- Corresponds to
id
property in the@Database
annotation. - Any value other than one that corresponds to the
id
property in the@Database
annotation is not allowed.
mode:
- Corresponds to one member in the list of
modes
, found in the@Database
annotation. - Any value other than one that corresponds to one of the
modes
listed in the@Database
annotation is not allowed.
source-table:
- Database table with "actual" data for the assert operation.
target:
- Refers to a data file, found in folder
data/
, a sibling to the ETLUnit test class.
target-file:
- Not supported.
source-schema:
- The source table's schema name.
sql-script:
- Store the SQL script in a file with the extension "
.sql
" in its name, in foldersql/
, a sibling to the ETLUnit test.
sql:
- In-line SQL script.
source-reference-file-type:
- Name of the FML file to be applied to the source table. Does not include extension "
.fml
."
target-reference-file-type:
- Name of the FML file to be applied to the target data file. Does not include extension "
.fml
."
reference-file-type:
- Synonym to
source-reference-file-type.
column-list:
- A list of column names to include or exclude.
column-list-mode:
- Valid values:
include
- Include for consideration the columns enumerated in column-list.
exclude
- Exclude for consideration the columns enumerated in column-list.
assertion-mode:
- Valid values:
equals
- Byte for byte, the source content matches the target content.
empty
- The source is empty.
notEmpty
- The source is not empty.
failure-id:
- In case of failure in the assert operation, use this failure id instead of the one that would normally be generated.
assert()
for File Context
[File Feature Module - Configuration properties not listed here.]
source-file:
- Required
- This is the "actual" data in the assert operation. It may come from one of two places:
- Output from the
execute
operation. - A file that has been registered with ETLUnit with the
register
operation.
- Output from the
sql:
- In-line SQL script.
sql-script:
- Store the SQL script in a file with the extension "
.sql
" in its name, in foldersql/
, a sibling to the ETLUnit test.
target:
- Refers to a data file, found in folder
data/
, a sibling to the ETLUnit test class.
classifier:
- Value is a string. Ad-hoc name of a classifier that will prevent, for instance, a registered file from producing an
ERR_REGISTER_FILE_NAME_DUPLICATED
error.
context:
- Used like classifier, it sets the assert in a different context than similar asserts. May prevent a registered file from producing an
ERR_REGISTER_FILE_NAME_DUPLICATED
error.
producer:
failure-id:
- In case of failure in the assert operation, use this failure id instead of the one that would normally be generated.
assertion-mode:
- Valid values:
equals
- - the default- Byte for byte, the source and target files compare favorably.
exists
- The source file exists
notExists
- The source file does not exist
empty
- The source file is empty
notEmpty
- The source file is not empty
reference-file-type:
- Synonym to
source-reference-file-type
.
base-source-reference-file-type:
- Reference file type for the source file in a three-tiered assertion (
source-file
,sql
orsql-script
, andtarget
). source-file
must have FML for this to work.- Example:
The variable $file-table-name
, below, is internal to ETLUnit. In this case, it points to the registered file, TEST
.
@JoinSuite(name:'file_register')
class file_register_iv
{
@Description(description: 'Copied from OptionsIntegrationTest, in etlunit-file.')
@Test
sqlPolymorphic()
{
register(
file: 'POLYM',
registered-name: 'TEST'
);
assert(
source-file: 'TEST',
target: 'TEST_SQL_POLY',
sql: "SELECT FILE_NAME_DATE, 'POP' || POS_TYPE || 'STAR' AS POP_STAR FROM $file-table-name",
base-source-reference-file-type: 'sqlPolymorphic',
source-reference-file-type: 'test_ii',
target-reference-file-type: 'test_ii'
);
}
}
source-reference-file-type:
- Name of the FML file to be applied to the source file. Does not include extension "
.fml
."
target-reference-file-type:
- Name of the FML file to be applied to the target file. Does not include extension "
.fml
."
column-list:
- A list of column names to include or exclude.
column-list-mode:
- Valid values:
include
- Include for consideration the columns enumerated in column-list.
exclude
- Exclude for consideration the columns enumerated in column-list.
assert()
for Data Sets Context
data-set-name:
- Required
- Matches the name of a text file in a folder '
dataset
,' which is a sibling of the ETLUnit test class. - The data set text file has file extention '
.dataset
.'
data-set-id:
- The
data-set-id
's value matches the value of theid
property in the data set member's properties section.
failure-id:
- The failure to be generated by ETLUnit if the operation using the data set fails.
- A failure elsewhere in the test method will not invoke this failure ID.
ignore-data-set-properties:
- Causes the ETLUnit test to ignore properties included in the data set member's properties section.
- This does not include the
id
property in the data set member's properties section.
Examples
Here is an example of the assert operation in the Database context that involves some configuration and FML.
Supporting Files
Configuration
- The POM is placed in the root of the project.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<dependencies>
<dependency>
<groupId>org.bitbucket.bradleysmithllc.etlunit</groupId>
<artifactId>etlunit-core</artifactId>
<version>${etlunit.project.version}</version>
</dependency>
<dependency>
<groupId>org.bitbucket.bradleysmithllc.etlunit</groupId>
<artifactId>etlunit-h2-database</artifactId>
<version>${etlunit.project.version}</version>
</dependency>
<dependency>
<groupId>org.bitbucket.bradleysmithllc.etlunit</groupId>
<artifactId>etlunit-cli</artifactId>
<version>${etlunit.project.version}</version>
</dependency>
<dependency>
<groupId>org.bitbucket.bradleysmithllc.etlunit</groupId>
<artifactId>etlunit-file</artifactId>
<version>${etlunit.project.version}</version>
</dependency>
</dependencies>
...
</project>
etlunit.json
may go under foldersrc/test/resources/config
.
{
"install-features": ["h2-database", "file"],
"features": {
"database": {
"database-definitions": {
"edw": {
"implementation-id": "h2",
"schema-scripts": [
"schema.ddl"
]
}
}
}
}
}
schema.ddl
might be placed undersrc/main/h2/edw
, in this case. The project in this example is using an h2 database engine, andedw
is the name of a declared database (above).
CREATE TABLE TEST_TABLE_VII
(
ID INT NOT NULL,
COL1 VARCHAR(256) NOT NULL,
COL2 INTEGER NOT NULL,
COL3 DATETIME NOT NULL,
COL4 VARCHAR(256) NOT NULL,
COL5 VARCHAR(256) NOT NULL,
COL6 VARCHAR(256) NOT NULL,
COL7 VARCHAR(256) NOT NULL,
PRIMARY KEY(ID)
);;
- r
eference-file-type-catalog.json
should be placed insrc/main/reference/file/fml
folder.
{
"reference-file-types":
{
"description": "Database types for the edw-pet data schema",
"packages":
{
"edw.public":
{
"description": "Tables in the package",
"types":
{
"TEST_TABLE_VII": {}
}
}
}
}
}
test_table_vii.json
in this case is placed insrc/main/reference/file/fml/edw/public
. TODO: Fix this link: SEE FML File Placement Options for more info.
{
"description": "Desc",
"default-version": "edw.1",
"versions": ["edw.1"]
}
id_date_mdy_stg.fml
is placed in the folder data/
, a sibling of the ETLUnit test class.
{
"flat-file" : {
"format-type" : "delimited",
"row-delimiter" : "\n",
"column-delimiter" : "\t",
"null-token" : null,
"columns" : [ {
"id" : "ID",
"type" : "INTEGER",
"length" : 10,
"basic-type" : "integer"
}, {
"id" : "COL3",
"type" : "TIMESTAMP",
"length" : 8,
"basic-type" : "string",
"format": "MMddyyyy"
} ],
"orderBy" : [ "ID" ]
}
}
Place file test_table_vii+edw.1.fml
in a location like src/main/reference/file/fml/edw/public
. TODO:Fix this link: Check out FML File Placement Options for more info.
{
"flat-file" : {
"format-type" : "delimited",
"row-delimiter" : "\n",
"column-delimiter" : "\t",
"null-token" : null,
"columns" : [ {
"id" : "ID",
"type" : "INTEGER",
"length" : 10,
"basic-type" : "integer"
}, {
"id" : "COL1",
"type" : "VARCHAR",
"length" : 256,
"basic-type" : "string",
"default-value": {"expression": "ID - ${rowData['ID']}"}
}, {
"id" : "COL2",
"type" : "INTEGER",
"length" : 10,
"basic-type" : "integer",
"default-value": {"initial-value": 10000, "increment": -3}
}, {
"id" : "COL3",
"type" : "TIMESTAMP",
"length" : 23,
"basic-type" : "string",
"format": "MMddyyyy",
"default-value": {"initial-value": 1012000}
}, {
"id" : "COL4",
"type" : "VARCHAR",
"length" : 256,
"basic-type" : "string",
"default-value": {"initial-value": 400}
}, {
"id" : "COL5",
"type" : "VARCHAR",
"length" : 256,
"basic-type" : "string",
"default-value": {"initial-value": 500}
}, {
"id" : "COL6",
"type" : "VARCHAR",
"length" : 256,
"basic-type" : "string",
"default-value": {"initial-value": 600}
}, {
"id" : "COL7",
"type" : "VARCHAR",
"length" : 256,
"basic-type" : "string",
"default-value": {"initial-value": 700}
} ],
"primaryKey" : [ "ID" ],
"orderBy" : [ "ID" ]
}
}
Data
/*-- ID COL3 --*/ /*-- INTEGER VARCHAR --*/ 1 01012014 2 02022014 3 03032014 4 04042014 5 05052014 6 06062014 7 07072014 8 08082014 9 09092014 10 10102014 11 11112014 12 12122014
Sample Test Code
@Description( description: 'Shows how to ignore columns on the assert')
@Database(id: 'edw')
class assert_with_columns_ii {
@Test
assert_many_against_few_with_implied_src_FML(){
stage(
source: 'ID_AND_DATE_X_MDY',
target-table: 'TEST_TABLE_VII',
reference-file-type: {
id: 'id_date_mdy_stg'
}
);
assert(
source-table: 'TEST_TABLE_VII',
target-reference-file-type: {
column-list-mode: 'include',
column-list: ['ID','COL3']
},
target: 'ID_AND_DATE_X_MDY'
);
}
}