Version:

Profiles 0.7.x Changelog

Changelog for Profiles v0.7.x.

  •   5 minute read  

  • Last modified: Aug 14, 2023

Version 0.7.3

14 August 2023

What’s New

  • pb show idstitcher-report:pb show idstitcher-report: By passing flag --id_stitcher_model, you can now create an HTML report with relevant results and graphics including largest cluster, ID graph, etc.
  • Material Registry has been updated to version 4, as additional information is now stored for target (as defined in siteconfig), system username, and invocation metadata (hostname and the project’s invocation folder). So now, if anyone logs into the system and creates material objects using PB, then these details will be stored. This is based on a feature request from one of our customers. Note: make sure to execute pb validate access for migrating the registry.
  • pb discover materials:pb discover materials - This command now shows a few additional columns - target, username, hostname, invocation folder.
  • Default ID stitcher: In the inputs file, the key to_default_stitcher needs to be set to true explicitly for an ID to get picked in the default ID stitcher. This field is optional and by default set to false, without impacting if the project is using a custom ID stitcher. In your project file, if you remove the key id_stitcher: models/<name of ID stitcher model>, then it’ll use the default ID stitcher and create a material view of the name <entity_name>_default_id_stitcher.
  • In the inputs.yaml file, table or view names now appear under a key named app_defaults:. This signifies that these are values that input defaults to, when the project is run directly. For library projects, inputs can be remapped and appdefaults overridden. when library projects are imported.
  • Schema has been migrated from 38 -> 40 in the project file.

Improvements

  • pb init pb-project:pb init pb-project: Added keys on default ID stitcher.
  • A few improvements behind the scenes, for enhancing the overall functionality.

Bug Fixes

  • Resolved the issue where projects migrated using migrate_on_load were referring to the location of the migrated project in the material registry. This was affecting the count of ID’s before and after stitching.
  • Fixed bug where ID stitcher wouldn’t check whether a material was actually existing in the database, before running in incremental mode.
  • When the material registry was on an unsupported common tables version, then the project environment loading would fail, thereby crashing the application. This has now been resolved.
  • Features defined in Python models, now do appear in the list of features.
  • Vars can still be specified in specs of a feature table model. However, the app ignores them. This is a bug and would be fixed in subsequent releases.

Version 0.7.2

24 July 2023

Our newest release brings enhanced functionality and a more efficient experience.

What’s New

  • Model Enable/Disable: You can now enable or disable specific models using the materialization key in model specifications. Use the status key to set values. For more information, refer to Models enabling themselves.
  • Migrate Auto: When migrating a project, the ordering of elements now remains the same as in the original files, preserving comments.
  • Graceful Application Exit: You can now exit the application gracefully while it’s running. For example, if you’re generating material tables using the run command, you can exit using Ctrl+C.
  • Schema Migration: The schema version in the project file has been updated from 37 to 38.

Improvements

  • Projects created using init pb-project now include dependencies.
  • Instead of generating one big SQL file, we now create multiple files in a folder during SQL generation of a feature table model. This reduces the disk space requirements.
  • Internal optimizations have been implemented to improve overall performance and efficiency.

Bug Fixes

  • An issue has been fixed where insufficient grants for accessing the warehouse would lead to duplicate suggested queries. Also, in some cases, incorrect queries were displayed, such as when a Redshift user was asked to grant a role.
  • The project URL is now being stored in the material registry, instead of GitHub passkey.
  • Fixed a bug where macros defined in a separate file as global macros were unable to access a common context.
  • Resolved a bug where Python models were not appearing in the dependency graph.

Version 0.7.1

23 June 2023

Our latest release addresses some critical issues in the previous release. Therefore, if you’re on v0.7.0, then it’s highly recommended to update to the latest version.

Version 0.7.0

22 June 2023

Our newest release is quite significant in terms of new features and improvements offered. Be sure to try it out and share your feedback with us.

What’s New

  • query - A new command which displays output of tables/views from the warehouse. So you can view generated material tables from the CLI itself. For example, pb query "select * from {{this.DeRef("models/user_id_stitcher")}}".
  • show idstitcher-report - A new sub command that creates report on an ID stitcher run. Such as, whether it converged, count of Pre-stitched ID’s before run, Post-stitched ID’s after run, etc. Usage: pb show idstitcher-report .
  • show user-lookup - A new sub command that allows you to search a user by using any of the traits as ID types. E.g., pb show user-lookup -v <trait value>.
  • If non-mandatory inputs required by the model are not present in the warehouse, you can still run the model. Applicable to packages and feature tables.
  • Schema updated from 33 -> 37 in the project file. Please note that the material registry has been migrated to version 3, so you’ll have to execute pb validate access once in order to execute the run command.

Improvements

  • Added an optional field source_metadata in the model file inputs.yaml.
  • Added EnableStatus field in materialization so that models can be enabled and disabled automatically based on whether it is required or not.
  • Default ID stitcher now supports incremental mode as well.
  • In macros, you can now specify timestamps in any format.

Bug Fixes

  • In case a project is migrated using flag migrate_on_load, then src_url in the material registry was pointing to the new folder. Now, that is fixed.
  • Resolved bugs in generating edges for dependency graphs.
  • Tons of several other improvements and bug fixes under the hood.

Questions? Contact us by email or on Slack