Change Log

Owned by Orcrist
Last updated: Aug 18, 2014
5 min read

Change log.  The latest version number is at the top.  If the version number is red, it is still a work in progress.

3.8.2

3.8.1

 - Fix for ETLUNIT-22

3.8.0

  • Fix for ETLUNIT-20
  • Fix for ETLUNIT-21
  • Fix for ETLUNIT-19

3.7.0

  • Added @Ignore tag for bypassing tests temporarily
  • Started updating this page again now that Michael is reading it.

3.6.0

- added 'not' and 'eor' to the test specification for compound statements.

 - added sorting for list reference file types command in the shell.

3.5.1

 - Implemented SqlServer instance names in informatica module - for relational connections.

 - Augmented informatica database integration test to work with 2K8 and 2K12 with instance names.

3.5.0

 - Added SQL server database instance names and AD authentication.

 - Added new integration tests against SqlServer 2K8 and 2K12 using instance names.

3.4.14

  - Changed Informatica relational connection names to begin with a double '_' to make cleanup easier.

3.4.13

  - Changed database names to begin with a double '_' to make automating cleanup easier.

 - Changed informatica folders to begin with a double '_' to make automating cleanup easier.

 

3.4.12

 

 - Minor miscellaneous fixes.

3.4.11

 - Fixed Sql Server error where it could not lock the model database.  Added a detection for the error and a retry.

3.4.10

- Fixed a problem with assertions and reference file types with columns.

3.4.9

 - Fixed informatica folder and workflow states.

3.4.8

- Fixed a bug where a Clob used as an order column would cause diffs to use the toString() representation of the clob rather than the actual text.

 - Add informatica check to see if folder exists before assuming it does.

3.4.7

 - Fixed a bug in FML default values where an index out of bounds could occur.

 - Fixed a race condition with assertions and temporary files.

3.4.6

 - enhance vrt so that it verifies the fml as well (load and verify).

3.4.5

 - bug fixes for vrt command.

 - Moved tags back to the target folder because having them persist was not going to be good.

3.4.4

 

Reference types:
 - use symlinks to reference tables imported from another source. E.G., STORE in pos-alx-mart is named pos-alx-mart-DBO-STORE but comes from edw-DBO-STORE
 - Provide a catalog of reference types exported from a project so they can be queried. Provide a cli command for exploring
 - Use a json file to describe an fml, with it's classifiers, versions, and current versions.
 - Create an extension fml so that it is trivial to extend an external fml but simply eliminate columns. This may be done this way, E.G.:
  stage(
   source: 'SOURCE',
   reference-file-type: {
      column-list-mode: 'include',
      column-list: [
        'STORE_SK'
      ]
   },
   target-table: 'STORE'
  )

 - Meta Data for reference types. Fix it since it was removed.

 

 

Bugs -
- Make reference file type catalogs resolve locally as well.
- Make the maven plugin and the shell resolve file types the same way

 -add verify command line tool for reference file types.

  -align executor counts and test numbers on console output

*E.G.,

*1: 1/10
*12: 10/10

* 1:  1/10
*12: 10/10

 - store tags in the user's home ~/.etlunit/tags/project folder rather than target

 - add variable value assertion and create test to detect context thread issue

 - database sql scripts process with velocity / context

 

3.4.2

 - Added meta data back for reference file types.

 - Added synonyms for command line tasks.  (Such as lrt for list-reference-types)

 - Fixed a bug where reference type catalogs weren't loaded from the local project.

3.4.1

 - Bug fixes for tags.

3.4.0

 - Implemented reference type manager

 - changes to assert

 - Added tags.

 - Added a cli command 'tag'

3.0.5

 - Bug fixes

3.0.4

 - Increased agent version to 3.3.0.1

3.0.3

 - Removed all usages of File.createTempFile in favor of managed temp files.

 - Fixed major display bugs on the better report page.

 - Fixed parallel execution problem in the status reporters.

 - Fixed race condition in set operation.

3.0.2

 - Fixed logging problem

3.0.1

 - Minor bugfix.

3.0.0

 - Introduced parallel execution.  Now a project can be configured like surefire to use a certain number of parallel executors.

2.0.31

 - Changed the file assertion to match the schema resolution rules of the other assertions.  First matches on the file attribute, then falls back to the target file name for the implicit matches.

2.0.30

 - Added format attribute to fml columns so the default timestamp format can be overridden.

 - Fixed a bug with the escape non printable attribute being preserved with a schema view materialization.

 - Added a hexidecimal data type for fmls.

2.0.29

 - Fixed a bug where an fml created via a view would produce an incompatible schema when the order or key columns differ from the column order.

 - Fixed bug with clobs and blobs reading from a file.

 

2.0.28

 - Made file paths case insensitive.

 - Associated errors with the correct log file (the method, not the class).

2.0.27

 - Corrected paths in the report file.

 - Made logs per-method not per class only.

2.0.26

 - Made the file paths in the report page relative urls.

 - Added FMLs to the meta data artifacts

2.0.25

- Made shorter temp file names so the length limitation wouldn't be hit as often (some of the names were getting quite long).

2.0.24

- Added sql extracts to file assertions - so now you can provide a sql statement or script to a file assertion.
- Fixed a small bug that printed extraneous 0's to the console.
- Preserved the row, column and null tokens when creating a view of a file schema in an assertion - database or file.

2.0.23

- Fixed schema views when sing assertions and primary keys

2.0.22

- Fixed agent-related bug

2.0.21

- Etlagent 2.2.0.3 which gives better temp file management and logging.

2.0.20

- Minor pom version.

2.0.19

- Added methods to runtime support around temp files.

2.0.18

- Updated to use etlagent 2.2.0.0 so the vfs can be used for file transfer.

2.0.17

- Minor tweak for old javacl-parser version.

2.0.16

- Fixed a bug in the clamshell scripts where it didn't update properly.
- Fixed clob implementation in the h2 data file.

2.0.15

- Fix for JDK 6. Removed a new attribute that was unintentionally referenced.

2.0.14

- H2 Data File Implementation. All local file operations now take place in a database table for performance and scalability. The consequence of this is that the fml's used are now constrained by actual types. E.G., If there is a primary key defined, your data set must follow it. If data types are defined for your table columns, the data you supply for the column must also match. This has been encouraged before, but now it will cause an error.
- Fml descriptors now support a scale attribute used for numeric and decimal types. The database module now populates the length and scale attributes for all database columns. This applies to new fml creation - existing fml's will be unmodified.
- Cached database metadata will cause a problem - do a clean test the first time you use this version.
- Using exclude columns in an assert which results in a 0-column intersection with the target fml is now an error, even if there are still columns present in the target fml.
- Using exclude and include lists on assertions is a little harder now since eliminating columns from the primary key might be problematic. E.G., your table has a pk on sk and date, but if you exclude sk, you are left with date which probably won't be unique. If this is a problem you should probably remove the primary key from your fml.