Version: Next

DataHub Releases

Summary

Version	Release Date	Links
v0.14.0.2	2024-08-21	Release Notes, View on GitHub
v0.14.0.1	2024-08-16	Release Notes, View on GitHub
v0.14.0	2024-08-13	Release Notes, View on GitHub
v0.13.3	2024-05-23	Release Notes, View on GitHub
v0.13.2	2024-04-16	View on GitHub
v0.13.1	2024-04-02	View on GitHub
v0.13.0	2024-02-29	View on GitHub
v0.12.1	2023-12-08	View on GitHub
v0.12.0	2023-10-25	View on GitHub
v0.11.0	2023-09-08	View on GitHub
v0.10.5	2023-08-02	View on GitHub
v0.10.4	2023-06-09	View on GitHub
v0.10.3	2023-05-25	View on GitHub
v0.10.2	2023-04-13	View on GitHub
v0.10.1	2023-03-23	View on GitHub
v0.10.0	2023-02-07	View on GitHub
v0.9.6.1	2023-01-31	View on GitHub
v0.9.6	2023-01-13	View on GitHub
v0.9.5	2022-12-23	View on GitHub
v0.9.4	2022-12-20	View on GitHub
v0.9.3	2022-11-30	View on GitHub
v0.9.2	2022-11-04	View on GitHub
v0.9.1	2022-10-31	View on GitHub
v0.9.0	2022-10-11	View on GitHub
v0.8.45	2022-09-23	View on GitHub
v0.8.44	2022-09-01	View on GitHub
v0.8.43	2022-08-09	View on GitHub
v0.8.42	2022-08-03	View on GitHub
v0.8.41	2022-07-15	View on GitHub

v0.14.0.2

Released on 2024-08-21 by @RyanHolstien.

Fixed issues

Kafka-Setup image was not being published in v0.14.0.1 due to the Kafka 3.7.0 library no longer being available which was fixed by this PR post release: https://github.com/datahub-project/datahub/commit/09acd5b989e3b00a5dce263c70cb27bcd744abb9

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.14.0.1...v0.14.0.2

v0.14.0.1

Released on 2024-08-16 by @RyanHolstien.

Hotfix release, for release notes see v0.14.0

Fixes issues:

Known issue with Kafka-Setup on new kubernetes deployments where topics would not get automatically created
Known issue with environment configuration from hashing algorithm configuration changes when configuring custom ElasticSearch document id hashing algorithm

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.14.0...v0.14.0.1

v0.14.0

Released on 2024-08-13 by @RyanHolstien.

Known Issues

Issue with kafka-setup missing a script for new deployments, hotfix will be released shortly

What's Changed

fix(ingest/unity-catalog) upstream lineage for hive_metastore external table with s3 location by @dushayntAW in https://github.com/datahub-project/datahub/pull/10546
feat(ingestion/looker): ingest explore tags into the DataHub by @sid-acryl in https://github.com/datahub-project/datahub/pull/10547
fix(instropection): fix configuration application order by @david-leifker in https://github.com/datahub-project/datahub/pull/10579
fix(ingest/slack): pull real names by @hsheth2 in https://github.com/datahub-project/datahub/pull/10565
fix(ingest): Remove env deprecation message by @treff7es in https://github.com/datahub-project/datahub/pull/10581
test(ingest/sql): refactor CLL generator + add tests by @hsheth2 in https://github.com/datahub-project/datahub/pull/10580
docs(remote-ingestion): update description and deployment instructions by @darnaut in https://github.com/datahub-project/datahub/pull/10574
fix(ingest): DataProcessInstance.emit_process_end() ignored start_timestamp_millis by @obaltian in https://github.com/datahub-project/datahub/pull/10539
fix(ingest/metabase): Fix for query template expressions and invalid URNs for Text Cards by @pulsar256 in https://github.com/datahub-project/datahub/pull/10381
feat(graphql): Support tagging incidents and assertions via GraphQL API by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10575
docs(update): updating-datahub by @david-leifker in https://github.com/datahub-project/datahub/pull/10585
docs: reorder semantics guide to the bottom by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10541
feat(auth): add viewTests platform privilege by @ksrinath in https://github.com/datahub-project/datahub/pull/10413
feat(ingestion/SageMaker): Remove deprecated apis and add stateful ingestion capability by @TonyOuyangGit in https://github.com/datahub-project/datahub/pull/10573
fix(search): fix autocomplete filter by @david-leifker in https://github.com/datahub-project/datahub/pull/10599
fix(ingest/snowflake): handle column level lineage for dbt temporary tables by @john-claro-cko in https://github.com/datahub-project/datahub/pull/10258
fix(mae-consumer): fix UpdateIndicesHook ignoring events with forceIndexing property set to true by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10586
feat(fieldpaths): prevent duplicate field paths by @david-leifker in https://github.com/datahub-project/datahub/pull/10590
docs: update Town Hall page by @maggiehays in https://github.com/datahub-project/datahub/pull/10588
fix(search): implement queryByDefault annotation for SearchableRef by @david-leifker in https://github.com/datahub-project/datahub/pull/10603
fix(ingest/sagemaker): remove unsupported config by @hsheth2 in https://github.com/datahub-project/datahub/pull/10606
feat(neo4j): combine neo4j statements in addEdge into one statement by @deepgarg-visa in https://github.com/datahub-project/datahub/pull/10598
feat(neo4j): improve neo4j read query performance by specifying labels by @deepgarg-visa in https://github.com/datahub-project/datahub/pull/10593
feat(ingest): fetch connections from the backend by @hsheth2 in https://github.com/datahub-project/datahub/pull/10511
feat(graphql): custom complexity calculator and separate configurable thread pool for graphQL by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10562
feat(ingest): enable stateful ingestion safety threshold by @hsheth2 in https://github.com/datahub-project/datahub/pull/10516
fix(ingest/spark): Bumping OpenLineage version to 0.14.0 by @treff7es in https://github.com/datahub-project/datahub/pull/10559
fix(ingest/dbt): only generate one subtype by @hsheth2 in https://github.com/datahub-project/datahub/pull/10615
fix(ingest/snowflake): make test connection logs less noisy by @hsheth2 in https://github.com/datahub-project/datahub/pull/10587
fix(ingest): move status aspect fixer logic by @hsheth2 in https://github.com/datahub-project/datahub/pull/10591
feat(data quality): update models, add assertions cli with snowflake integration by @mayurinehate in https://github.com/datahub-project/datahub/pull/10602
fix(gms/autosuggestion): autosuggestion query not returning the result if the query text has a prefix or suffix '-' on the search field by @siladitya2 in https://github.com/datahub-project/datahub/pull/10512
feat(consumers): mce-consumer throttling based on mae-consumer lag by @david-leifker in https://github.com/datahub-project/datahub/pull/10626
Add support for runAssertion, runAssertions, and runAssertionsForAsset APIs by @noggi in https://github.com/datahub-project/datahub/pull/10605
feat(graphql) data contract resolvers for graphql by @jayacryl in https://github.com/datahub-project/datahub/pull/10618
Revert "feat(graphql) data contract resolvers for graphql" by @jayacryl in https://github.com/datahub-project/datahub/pull/10631
fix(views): Add relationship annotation to GlobalViewsSettings urn by @pedro93 in https://github.com/datahub-project/datahub/pull/10597
feat(cli) Delete form references when using delete CLI by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10629
feat(ingest/looker): add ownership info to independent looks by @k7ragav in https://github.com/datahub-project/datahub/pull/10624
log(custom-plugins): add additional logging for spring plugins by @david-leifker in https://github.com/datahub-project/datahub/pull/10627
refactor(ui/glossary): Clean up term deletion by @asikowitz in https://github.com/datahub-project/datahub/pull/10589
fix(views): handle unknown view when resolving a view to a filter by @darnaut in https://github.com/datahub-project/datahub/pull/10640
feat(lineage): change query structure for explored hop limit by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10607
feat(ingest): measure sink bottlenecking by @hsheth2 in https://github.com/datahub-project/datahub/pull/10628
fix(ingest/iceberg): update iceberg source to support newer versions of pyiceberg at runtime by @cccs-eric in https://github.com/datahub-project/datahub/pull/10614
feat(ingest/redshift): Adding way to filter s3 paths in Redshift Source by @treff7es in https://github.com/datahub-project/datahub/pull/10622
feat(businessAttribute): parallelize-business-attribute-propagation by @deepgarg-visa in https://github.com/datahub-project/datahub/pull/10638
docs(ingest): remove trailing comma on athena permission by @nephtyws in https://github.com/datahub-project/datahub/pull/10634
doc(roles): update privileges by @ksrinath in https://github.com/datahub-project/datahub/pull/10528
docs(subscriptions): adding docs for assertion level subscriptions on managed DH by @jayacryl in https://github.com/datahub-project/datahub/pull/10495
feat(ingest): add fast query fingerprinting by @hsheth2 in https://github.com/datahub-project/datahub/pull/10619
fix(ingestion/airflow-plugin): updated the document for developers by @dushayntAW in https://github.com/datahub-project/datahub/pull/10633
fix(ingest/trino): variable reference before define by @anshbansal in https://github.com/datahub-project/datahub/pull/10646
feat(entity-client): restli batchGetV2 batchSize fix and concurrency by @david-leifker in https://github.com/datahub-project/datahub/pull/10630
docs(): Adding API docs for incidents, operations, and assertions by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10522
feat(ci): fix conditionals and consolidate change detection by @david-leifker in https://github.com/datahub-project/datahub/pull/10649
fix(ingest/snowflake): avoid overfetching schemas from datahub by @hsheth2 in https://github.com/datahub-project/datahub/pull/10527
docs: add note for subResourceType being a fieldPath by @anshbansal in https://github.com/datahub-project/datahub/pull/10660
fix(ingest/qlik): improve logging for debug by @anshbansal in https://github.com/datahub-project/datahub/pull/10659
fix(doc): Fix doc typo in transformer by @sid-acryl in https://github.com/datahub-project/datahub/pull/10658
feat(graphql) data contract resolvers by @jayacryl in https://github.com/datahub-project/datahub/pull/10632
fix(openapiv3): v3 scroll response fix by @david-leifker in https://github.com/datahub-project/datahub/pull/10654
Use type: string for enum schemas by @kevin1chun in https://github.com/datahub-project/datahub/pull/10663
fix(ingestion/airflow-plugin): airflow remove old tasks by @dushayntAW in https://github.com/datahub-project/datahub/pull/10485
feat(platform): added db2 platform by @pankajmahato-visa in https://github.com/datahub-project/datahub/pull/10601
feat(ingestion/kafka)-Add support for ingesting schemas from schema registry by @aabharti-visa in https://github.com/datahub-project/datahub/pull/10612
fix(azure_ad): print request URL on error by @darnaut in https://github.com/datahub-project/datahub/pull/10677
docs(ingest): Rename csv / s3 / file source and sink by @asikowitz in https://github.com/datahub-project/datahub/pull/10675
feat(ingest/glue): database parameters extraction by @skrydal in https://github.com/datahub-project/datahub/pull/10665
fix(azure_ad): fix infinite loop on request error by @darnaut in https://github.com/datahub-project/datahub/pull/10679
perf(ingestion/fivetran): Connector performance optimization by @shubhamjagtap639 in https://github.com/datahub-project/datahub/pull/10556
feat(ingest): make query formatting more robust by @hsheth2 in https://github.com/datahub-project/datahub/pull/10678
feat(cli) Add actors to forms yaml API by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10683
doc(glossary): add note for github action for glossary by @anshbansal in https://github.com/datahub-project/datahub/pull/10687
feat(cli/data product): add support for institutional memory by @anshbansal in https://github.com/datahub-project/datahub/pull/10686
docs(cli/dataset): add dataset CLI get and upsert examples by @anshbansal in https://github.com/datahub-project/datahub/pull/10688
feat(ingest/airflow): fix materialize_iolets bug by @hsheth2 in https://github.com/datahub-project/datahub/pull/10613
feat(ingest/dbt): include package_name in dbt custom props by @hsheth2 in https://github.com/datahub-project/datahub/pull/10652
feat(ingest): add snowflake-summary source by @hsheth2 in https://github.com/datahub-project/datahub/pull/10642
feat(ui): Display 'View in Gitlab' if externalUrl is a link to Gitlab by @k7ragav in https://github.com/datahub-project/datahub/pull/10668
feat(ingest/cli): optionally show server config by @anshbansal in https://github.com/datahub-project/datahub/pull/10676
fix(docs): structured properties openapi guide by @david-leifker in https://github.com/datahub-project/datahub/pull/10671
docs(): Announcing DataHub Open Assertions Specification by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10609
fix(metadata-models) bridge gaps between graphql and pegasus models by @jayacryl in https://github.com/datahub-project/datahub/pull/10692
Aspect refs inside entity schema are nullable by @kevin1chun in https://github.com/datahub-project/datahub/pull/10695
feat(properties) Support custom properties on all entities with profile page by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10680
fix: APPT-43 | Lineage Edit: Modal Autocomplete by @sumitappt in https://github.com/datahub-project/datahub/pull/10569
chore(ui/ingest): improve description of executor ID by @darnaut in https://github.com/datahub-project/datahub/pull/10698
fix(ingest/fivetran): fix fivetran bigquery support by @hsheth2 in https://github.com/datahub-project/datahub/pull/10693
fix(ingest): fix redshift query urns + reduce memory usage by @hsheth2 in https://github.com/datahub-project/datahub/pull/10691
fix(operations): fix authorizer on operations controller by @david-leifker in https://github.com/datahub-project/datahub/pull/10701
fix(graphql): fix plugin collection by @david-leifker in https://github.com/datahub-project/datahub/pull/10696
fix(ingest/bigquery): Map BigQuery policy tags to datahub column-level tags by @sagar-salvi-apptware in https://github.com/datahub-project/datahub/pull/10669
fix(ingest/kafka-connect): Add lineage extraction for BigQuery Sink Connector in Kafka Connect source by @sagar-salvi-apptware in https://github.com/datahub-project/datahub/pull/10647
fix(search): fixes issue where exact match exclusive flag broke quoted structured search by @nmbryant in https://github.com/datahub-project/datahub/pull/10690
feat(openapi): openapi v3 updates by @david-leifker in https://github.com/datahub-project/datahub/pull/10710
fix(ingestion/sigma): Fix multiple requests http errors by @shubhamjagtap639 in https://github.com/datahub-project/datahub/pull/10616
docs(ingest): Add Oracle prerequisites by @darnaut in https://github.com/datahub-project/datahub/pull/10712
feat(gms): add ingestProposalBatch endpoint by @hsheth2 in https://github.com/datahub-project/datahub/pull/10706
feat(ingest/snowflake): refactor + parallel schema extraction by @hsheth2 in https://github.com/datahub-project/datahub/pull/10653
Expose get_entities_v2 endpoint in python client by @noggi in https://github.com/datahub-project/datahub/pull/10694
fix(docs): formatting of transformers code blocks by @walter9388 in https://github.com/datahub-project/datahub/pull/10670
feat(ingest/vertica): use 3 part naming by @Rajasekhar-Vuppala in https://github.com/datahub-project/datahub/pull/10636
feat(ingest): log http request retries by @hsheth2 in https://github.com/datahub-project/datahub/pull/10715
fix(ingestion/bigquery): user exceeded quota for concurrent project.lists requests by @shubhamjagtap639 in https://github.com/datahub-project/datahub/pull/10578
fix(ingest): fix dagster plugin release process by @hsheth2 in https://github.com/datahub-project/datahub/pull/10713
docs: add customer stories page by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10600
feat(ingest/bigquery): Support for View Labels by @ethan-cartwright in https://github.com/datahub-project/datahub/pull/10648
feat(observe) expose assertion runId and lastObservedMillis to graphql by @jayacryl in https://github.com/datahub-project/datahub/pull/10726
fix(ingest): pin numpy<2 for classification by @hsheth2 in https://github.com/datahub-project/datahub/pull/10725
feat(ingest/bigquery): support using table read permission without profiling by @hsheth2 in https://github.com/datahub-project/datahub/pull/10699
fix(ingest/looker): fix looker browse paths v2 by @hsheth2 in https://github.com/datahub-project/datahub/pull/10700
feat(strucutred-properties): structured properties delete and schema change support by @david-leifker in https://github.com/datahub-project/datahub/pull/10711
feat(ingest/snowflake): support more than 10k views in a db by @hsheth2 in https://github.com/datahub-project/datahub/pull/10718
feat(cli): Make ingest deploy create recipe with urn if not exists by @pedro93 in https://github.com/datahub-project/datahub/pull/10724
fix(ingestion/airflow-plugin): fixed the failing pipeline by @dushayntAW in https://github.com/datahub-project/datahub/pull/10737
chore(security): updates for security vulnerabilities by @david-leifker in https://github.com/datahub-project/datahub/pull/10740
fix(ingest/dbt): support emitting only model performance by @hsheth2 in https://github.com/datahub-project/datahub/pull/10714
config(header): increase header size to 32k by @david-leifker in https://github.com/datahub-project/datahub/pull/10743
chore(security): bump jetty version by @darnaut in https://github.com/datahub-project/datahub/pull/10744
feat(ingest/snowflake): log queries at info level by @hsheth2 in https://github.com/datahub-project/datahub/pull/10745
feat: allow task ownership as group by @fjmacagno in https://github.com/datahub-project/datahub/pull/10742
fix(ingest/logging): fix excessive ingestion logging by @pie1nthesky in https://github.com/datahub-project/datahub/pull/10735
docs(notifications): add personal notifications docs by @eboneil in https://github.com/datahub-project/datahub/pull/10730
feat(ci): update base requirements file by @anshbansal in https://github.com/datahub-project/datahub/pull/10747
feat(ui/data-contract): Data contract UI under Validation Tab by @amit-apptware in https://github.com/datahub-project/datahub/pull/10625
fix(ingestion/airflow-plugin): emit browsePathV2 by @dushayntAW in https://github.com/datahub-project/datahub/pull/10738
fix(ingest/tableau): warn with better error message by @anshbansal in https://github.com/datahub-project/datahub/pull/10749
fix(mae-consumer-job): add PE processor to component scan by @pankajmahato-visa in https://github.com/datahub-project/datahub/pull/10751
chore(alpine): update alpine base image by @david-leifker in https://github.com/datahub-project/datahub/pull/10754
fix(bigquery): use get() instead of hassattr for view labels by @ethan-cartwright in https://github.com/datahub-project/datahub/pull/10756
ci(ui): Add prettier to CI by @asikowitz in https://github.com/datahub-project/datahub/pull/10741
feat(spark): Adding OpenLineage symlink support to Spark lineage by @treff7es in https://github.com/datahub-project/datahub/pull/10637
fix(ingest/iceberg): add support for nested dictionaries when configuring pyiceberg by @cccs-eric in https://github.com/datahub-project/datahub/pull/10762
fix(media-type): fix proxy media-type and openapi patch endpoint by @david-leifker in https://github.com/datahub-project/datahub/pull/10763
test(mae-consumer): test for injection of pe-consumer by @david-leifker in https://github.com/datahub-project/datahub/pull/10755
fix(spark):Add option to disable symlink resolution by @treff7es in https://github.com/datahub-project/datahub/pull/10767
feat(openapi): restore Timeline OpenAPIv1 and deprecations by @david-leifker in https://github.com/datahub-project/datahub/pull/10768
feat(web-react): adds the possibility to track events through GA4 by @PatrickfBraz in https://github.com/datahub-project/datahub/pull/8231
docs: merge cli guide by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10464
feat(data quality): custom assertions models, graphql, sdk by @mayurinehate in https://github.com/datahub-project/datahub/pull/10761
chore(lint): spotless apply by @david-leifker in https://github.com/datahub-project/datahub/pull/10779
feat(protobuf): disable binary protoc custom properties by @david-leifker in https://github.com/datahub-project/datahub/pull/10778
feat(docs): Adding docs for custom assertion reporting APIs (WIP) by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10656
feat(schema-registry): enable config endpoint internal schema registry by @david-leifker in https://github.com/datahub-project/datahub/pull/10776
fix(ingest): use more aggressive errors with sqlglot by @hsheth2 in https://github.com/datahub-project/datahub/pull/10769
feat(ingest/snowflake): performance improvements by @hsheth2 in https://github.com/datahub-project/datahub/pull/10746
feat(ingest): add async batch mode to the rest sink by @hsheth2 in https://github.com/datahub-project/datahub/pull/10733
feat(search): adjust search config by @david-leifker in https://github.com/datahub-project/datahub/pull/10774
fix(ui/entityProfile/dataset): Show view definition tab if viewProperties.logic defined by @asikowitz in https://github.com/datahub-project/datahub/pull/10777
fix(ingest/snowflake): fix column batcher by @hsheth2 in https://github.com/datahub-project/datahub/pull/10781
docs(acryl cloud): release 0.3.3 by @anshbansal in https://github.com/datahub-project/datahub/pull/10772
build(deps-dev): bump vite from 4.5.2 to 4.5.3 in /datahub-web-react by @dependabot in https://github.com/datahub-project/datahub/pull/10199
docs(champions): update champions entry by @bossenti in https://github.com/datahub-project/datahub/pull/10721
feat(ingest): bump sqlglot by @hsheth2 in https://github.com/datahub-project/datahub/pull/10770
fix(ui/data-contract): fix freshness & schema assertion is not working by @amit-apptware in https://github.com/datahub-project/datahub/pull/10795
refactor(tags): Use TagUrn class when generating urn by @eboneil in https://github.com/datahub-project/datahub/pull/10786
fix(ingest/looker): prevent bad input fields by @hsheth2 in https://github.com/datahub-project/datahub/pull/10785
fix(ingest): add status aspect to dataProcessInstance by @hsheth2 in https://github.com/datahub-project/datahub/pull/10757
fix(ingest/pipeline): catch pipeline exceptions by @pie1nthesky in https://github.com/datahub-project/datahub/pull/10753
chore(gradle): remove httpclient 4 references by @david-leifker in https://github.com/datahub-project/datahub/pull/10787
feat(structuredproperties): aggregration fix & docs by @david-leifker in https://github.com/datahub-project/datahub/pull/10780
feat(ingest): set pipeline name in system metadata by @hsheth2 in https://github.com/datahub-project/datahub/pull/10190
fix(ingest/snowflake): add limits on tables/columns/queries in lineage by @hsheth2 in https://github.com/datahub-project/datahub/pull/10804
fix(airflow): fix airflow snowflake tests by @hsheth2 in https://github.com/datahub-project/datahub/pull/10803
feat(custom-plugins): improve plugin factory merge by @david-leifker in https://github.com/datahub-project/datahub/pull/10796
fix(ingest/snowflake): fix error case in column lineage by @hsheth2 in https://github.com/datahub-project/datahub/pull/10808
build(deps): bump braces from 3.0.2 to 3.0.3 in /docs-website by @dependabot in https://github.com/datahub-project/datahub/pull/10681
feat(ui): display chart query if it exists by @ngamanda in https://github.com/datahub-project/datahub/pull/10672
docs: update api overview by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10543
refactor(web-react): add encoder to support non-ASCII characters csv download by @PeamThom in https://github.com/datahub-project/datahub/pull/10496
fix(docs) adding dataset column tags docs by @eboneil in https://github.com/datahub-project/datahub/pull/10479
build(deps): bump ejs from 3.1.9 to 3.1.10 in /datahub-web-react by @dependabot in https://github.com/datahub-project/datahub/pull/10417
fix(metadata-service): consider missing entities in form assignment hook by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10392
feat(ingest/powerbi): powerbi dataset profiling by @looppi in https://github.com/datahub-project/datahub/pull/9355
fix(ui): show external url also in entity profile of containers by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10390
fix(ERModelRelationship) UUID should mimic datahub_guid.py by @rtekal in https://github.com/datahub-project/datahub/pull/10355
chore(vulnerability): Inefficient Regular Expression - Potential high time complexity leading to ReDoS by @Sukeerthi31 in https://github.com/datahub-project/datahub/pull/10315
chore(vulnerability): Bumped up reactour version to address high vulnerability by @pinakipb2 in https://github.com/datahub-project/datahub/pull/10218
build(deps): bump express from 4.18.2 to 4.19.2 in /docs-website by @dependabot in https://github.com/datahub-project/datahub/pull/10128
feat(backend): Add new PDL entities + models for persona capture by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/9637
feat(logging): unified request logging (graphql, openapi, restli) by @david-leifker in https://github.com/datahub-project/datahub/pull/10802
docs: hivePlatformAlias is different by @ssilb4 in https://github.com/datahub-project/datahub/pull/10765
fix(ingestion): ingest emails as empty if no ldap attribute by @tkdrahn in https://github.com/datahub-project/datahub/pull/9433
fix(patch): consider escaped characters when applying JSON patches by @ipolding-cais in https://github.com/datahub-project/datahub/pull/10717
fix(plugin): include ancestors when loading Spring custom plugin by @david-leifker in https://github.com/datahub-project/datahub/pull/10809
feat(docker/quickstart): Adding in support for overriding the conflue… by @brbrown25 in https://github.com/datahub-project/datahub/pull/10533
feat(ui): Add support for structured reporting of warnings and failures in the UI ingestion flow (ingest uplift 2/2) by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10790
docs(classification): correct the casing for full name infotype by @mayurinehate in https://github.com/datahub-project/datahub/pull/10782
feat(ingest): add and use file system abstraction in file source by @simaov in https://github.com/datahub-project/datahub/pull/8415
feat(ingest/lookml): ingest field tags by @sid-acryl in https://github.com/datahub-project/datahub/pull/10792
fix(assertions): minor changes in custom assertion api by @mayurinehate in https://github.com/datahub-project/datahub/pull/10794
build(jar): add datahub-custom-plugin-lib to jar workflow by @david-leifker in https://github.com/datahub-project/datahub/pull/10812
feat(SDK) Add FormPatchBuilder in python sdk and provide sample CRUD files by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10821
docs(ingest): add business glossary examples by @eboneil in https://github.com/datahub-project/datahub/pull/9851
feat(forms) Handle deleting forms references when hard deleting forms by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10820
refactor(ui): Misc improvements to the setup ingestion flow (ingest uplift 1/2) by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10764
fix(ingestion/airflow-plugin): pipeline tasks discoverable in search by @dushayntAW in https://github.com/datahub-project/datahub/pull/10819
feat(ingest/transformer): tags to terms transformer by @sagar-salvi-apptware in https://github.com/datahub-project/datahub/pull/10758
fix(ingestion/unity-catalog): fixed issue with profiling with GE turned on by @dushayntAW in https://github.com/datahub-project/datahub/pull/10752
feat(forms) Add java SDK for form entity PATCH + CRUD examples by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10822
feat(SDK) Add java SDK for structuredProperty entity PATCH + CRUD examples by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10823
feat(SDK) Add StructuredPropertyPatchBuilder in python sdk and provide sample CRUD files by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10824
feat(forms) Add CRUD endpoints to GraphQL for Form entities by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10825
add flag for includeSoftDeleted in scroll entities API by @kevin1chun in https://github.com/datahub-project/datahub/pull/10831
feat(deprecation) Return actor entity with deprecation aspect by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10832
feat(structuredProperties) Add CRUD graphql APIs for structured property entities by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10826
add scroll parameters to openapi v3 spec by @kevin1chun in https://github.com/datahub-project/datahub/pull/10833
fix(ingest): correct profile_day_of_week implementation by @jordanjeremy in https://github.com/datahub-project/datahub/pull/10818
feat(ingest/glue): allow ingestion of empty databases from Glue by @skrydal in https://github.com/datahub-project/datahub/pull/10666
feat(cli): add more details to get cli by @anshbansal in https://github.com/datahub-project/datahub/pull/10815
fix(ingestion/glue): ensure date formatting works on all platforms for aws glue by @sagar-salvi-apptware in https://github.com/datahub-project/datahub/pull/10836
fix(ingestion): fix datajob patcher by @david-leifker in https://github.com/datahub-project/datahub/pull/10827
fix(smoke-test): add suffix in temp file creation by @sid-acryl in https://github.com/datahub-project/datahub/pull/10841
feat(ingest/glue): add helper method to permit user or group ownership by @aviv-julienjehannet in https://github.com/datahub-project/datahub/pull/10784
Show data platform instances in policy modal if they are set on the policy by @githendrik in https://github.com/datahub-project/datahub/pull/10645
docs(patch): add patch documentation for how implementation works by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10010
fix(jar): add missing custom-plugin-jar task by @david-leifker in https://github.com/datahub-project/datahub/pull/10847
fix: also check exceptions/stack trace when filtering log messages by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10391
Update posts.md by @chardaway in https://github.com/datahub-project/datahub/pull/9893
chore(ingest): update acryl-datahub-classify version by @cburroughs in https://github.com/datahub-project/datahub/pull/10844
refactor(ingest): Refactor structured logging to support infos, warnings, and failures structured reporting to UI by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10828
fix(restli): log aspect-not-found as a warning rather than as an error by @ksrinath in https://github.com/datahub-project/datahub/pull/10834
fix(ingest/nifi): remove duplicate upstream jobs by @mayurinehate in https://github.com/datahub-project/datahub/pull/10849
fix(smoke-test): test access to create/revoke personal access tokens by @ksrinath in https://github.com/datahub-project/datahub/pull/10848
fix(smoke-test): missing test for move domain by @Kunal-kankriya in https://github.com/datahub-project/datahub/pull/10837
ci: update usernames to not considered for community by @anshbansal in https://github.com/datahub-project/datahub/pull/10851
env: change defaults for data contract visibility by @shirshanka in https://github.com/datahub-project/datahub/pull/10854
fix(ingest/tableau): quote special characters in external URL by @ipolding-cais in https://github.com/datahub-project/datahub/pull/10842
fix/ added click to perticular result and delay to element visibility by @Kunal-kankriya in https://github.com/datahub-project/datahub/pull/10861
ci(ingest): pin dask dependency for feast by @mayurinehate in https://github.com/datahub-project/datahub/pull/10865
fix(ingestion/lookml): liquid template resolution and view-to-view cll by @sid-acryl in https://github.com/datahub-project/datahub/pull/10542
feat(ingest/audit): add client id and version in system metadata props by @anshbansal in https://github.com/datahub-project/datahub/pull/10829
chore(ingest): Mypy 1.10.1 pin by @treff7es in https://github.com/datahub-project/datahub/pull/10867
docs: use acryl-datahub-actions as expected python package to install by @aviv-julienjehannet in https://github.com/datahub-project/datahub/pull/10852
docs: add new js snippet by @hsheth2 in https://github.com/datahub-project/datahub/pull/10846
refactor(ingestion): remove company domain for security reason by @shubhamjagtap639 in https://github.com/datahub-project/datahub/pull/10839
fix(ingestion/spark): Platform instance and column level lineage fix by @treff7es in https://github.com/datahub-project/datahub/pull/10843
feat(ingestion/tableau): optionally ingest multiple sites and create site containers by @haeniya in https://github.com/datahub-project/datahub/pull/10498
fix(ingestion/looker): Add sqlglot dependency and remove unused sqlparser by @sid-acryl in https://github.com/datahub-project/datahub/pull/10874
fix(manage-tokens): fix manage access token policy by @david-leifker in https://github.com/datahub-project/datahub/pull/10853
Batch get entity endpoints by @kevin1chun in https://github.com/datahub-project/datahub/pull/10880
feat(system): support conditional write semantics by @david-leifker in https://github.com/datahub-project/datahub/pull/10868
fix(build): upgrade vercel builds to Node 20.x by @hsheth2 in https://github.com/datahub-project/datahub/pull/10890
feat(ingest/lookml): shallow clone repos by @hsheth2 in https://github.com/datahub-project/datahub/pull/10888
fix(ingest/looker): add missing dependency by @hsheth2 in https://github.com/datahub-project/datahub/pull/10876
fix(ingest): only populate audit stamps where accurate by @hsheth2 in https://github.com/datahub-project/datahub/pull/10604
fix(ingest/dbt): always encode tag urns by @hsheth2 in https://github.com/datahub-project/datahub/pull/10799
fix(ingest/redshift): handle multiline alter table commands by @hsheth2 in https://github.com/datahub-project/datahub/pull/10727
fix(ingestion/looker): column name missing in explore by @sid-acryl in https://github.com/datahub-project/datahub/pull/10892
fix(lineage) Fix lineage source/dest filtering with explored per hop limit by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10879
feat(conditional-writes): misc updates and fixes by @david-leifker in https://github.com/datahub-project/datahub/pull/10901
feat(ci): update outdated action by @anshbansal in https://github.com/datahub-project/datahub/pull/10899
feat(rest-emitter): adding async flag to rest emitter by @gabe-lyons in https://github.com/datahub-project/datahub/pull/10902
feat(ingest): add snowflake-queries source by @hsheth2 in https://github.com/datahub-project/datahub/pull/10835
fix(ingest): improve auto_materialize_referenced_tags_terms error handling by @hsheth2 in https://github.com/datahub-project/datahub/pull/10906
docs: add new company to adoption list by @shtephlee in https://github.com/datahub-project/datahub/pull/10909
refactor(redshift): Improve redshift error handling with new structured reporting system by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10870
feat(ui) Finalize support for all entity types on forms by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10915
Index ExecutionRequestResults status field by @noggi in https://github.com/datahub-project/datahub/pull/10811
feat(ingest): grafana connector by @anshbansal in https://github.com/datahub-project/datahub/pull/10891
fix(gms) Add Form entity type to EntityTypeMapper by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10916
feat(dataset): add support for external url in Dataset by @dragontail in https://github.com/datahub-project/datahub/pull/10877
docs(saas-overview) added missing features to observe section by @jayacryl in https://github.com/datahub-project/datahub/pull/10913
fix(ingest/spark): Fixing Micrometer warning by @treff7es in https://github.com/datahub-project/datahub/pull/10882
fix(structured properties): allow application of structured properties without schema file by @gabe-lyons in https://github.com/datahub-project/datahub/pull/10918
fix(data-contracts-web) handle other schedule types by @jayacryl in https://github.com/datahub-project/datahub/pull/10919
fix(ingestion/tableau): human-readable message for PERMISSIONS_MODE_SWITCHED error by @sid-acryl in https://github.com/datahub-project/datahub/pull/10866
feat(ingest/snowflake): Add feature flag for view defintions by @ethan-cartwright in https://github.com/datahub-project/datahub/pull/10914
feat(ingest/BigQuery): refactor+parallelize dataset metadata extraction by @mayurinehate in https://github.com/datahub-project/datahub/pull/10884
fix(airflow): add error handling around render_template() by @hsheth2 in https://github.com/datahub-project/datahub/pull/10907
feat(ingestion/sqlglot): add optional default_dialect parameter to sqlglot lineage by @nadavgross in https://github.com/datahub-project/datahub/pull/10830
feat(mcp-mutator): new mcp mutator plugin by @david-leifker in https://github.com/datahub-project/datahub/pull/10904
fix(ingest/bigquery): changes helper function to decode unicode scape sequences by @PatrickfBraz in https://github.com/datahub-project/datahub/pull/10845
feat(ingest/postgres): fetch table sizes for profile by @pie1nthesky in https://github.com/datahub-project/datahub/pull/10864
feat(ingest/abs): Adding azure blob storage ingestion source by @joelmataKPN in https://github.com/datahub-project/datahub/pull/10813
fix(ingest/redshift): reduce severity of SQL parsing issues by @hsheth2 in https://github.com/datahub-project/datahub/pull/10924
fix(build): fix lint fix web react by @anshbansal in https://github.com/datahub-project/datahub/pull/10896
fix(ingest/bigquery): handle quota exceeded for project.list requests by @sagar-salvi-apptware in https://github.com/datahub-project/datahub/pull/10912
feat(ingest): report extractor failures more loudly by @hsheth2 in https://github.com/datahub-project/datahub/pull/10908
feat(ingest/snowflake): integrate snowflake-queries into main source by @hsheth2 in https://github.com/datahub-project/datahub/pull/10905
fix(ingest): fix docs build by @hsheth2 in https://github.com/datahub-project/datahub/pull/10926
fix(ingest/snowflake): fix test connection by @hsheth2 in https://github.com/datahub-project/datahub/pull/10927
fix(ingest/lookml): add view load failures to cache by @hsheth2 in https://github.com/datahub-project/datahub/pull/10923
docs(slack) overhauled setup instructions and screenshots by @jayacryl in https://github.com/datahub-project/datahub/pull/10922
fix(airflow): Add comma parsing of owners to DataJobs by @eboneil in https://github.com/datahub-project/datahub/pull/10903
fix(entityservice): fix merging sideeffects by @david-leifker in https://github.com/datahub-project/datahub/pull/10937
feat(ingest): Support System Ingestion Sources, Show and hide system ingestion sources with Command-S by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10938
chore() Set a default lineage filtering end time on backend when a start time is present by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10925
Added relationships APIs to V3. Added these generic APIs to V3 swagger doc. by @ajoymajumdar in https://github.com/datahub-project/datahub/pull/10939
docs: add learning center to docs by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10921
doc: Update hubspot form id by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10943
chore(airflow): add python 3.11 w/ Airflow 2.9 to CI by @hsheth2 in https://github.com/datahub-project/datahub/pull/10941
fix(Ingestor/Glue): Implement column upstream lineage between S3 and Glue by @sagar-salvi-apptware in https://github.com/datahub-project/datahub/pull/10895
fix(ingest/abs): Splitting abs utils into multiple files to not include abs specific includes which broke path_spec includes by @treff7es in https://github.com/datahub-project/datahub/pull/10945
fix(ingestion/looker): fix doc for sql parsing documentation by @sid-acryl in https://github.com/datahub-project/datahub/pull/10883
fix(ingest/bigquery): Adding missing BigQuery types by @treff7es in https://github.com/datahub-project/datahub/pull/10950
fix(ingest/setup): Fix for feast and abs source setup by @treff7es in https://github.com/datahub-project/datahub/pull/10951
fix(connections) Harden adding /gms to connections in backend by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10942
feat(siblings) Add flag to prevent combining siblings in the UI by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10952
fix(docs): make graphql doc gen more automated by @hsheth2 in https://github.com/datahub-project/datahub/pull/10953
feat(ingest/athena): Add option for Athena partitioned profiling by @treff7es in https://github.com/datahub-project/datahub/pull/10723
fix(spark-lineage): default timeout for future responses by @deepgarg-visa in https://github.com/datahub-project/datahub/pull/10947
feat(datajob/flow): add environment filter using info aspects by @anshbansal in https://github.com/datahub-project/datahub/pull/10814
fix(ui/ingest): ingest tab should show with manage ingestion privilege by @anshbansal in https://github.com/datahub-project/datahub/pull/10483
feat(ingest/looker): include dashboard urns in browse v2 by @hsheth2 in https://github.com/datahub-project/datahub/pull/10955
add a structured type to batchGet in OpenAPI V3 spec by @kevin1chun in https://github.com/datahub-project/datahub/pull/10956
fix(ui) Fix scroll on the domain sidebar to show all domains by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10966
fix: resolve incorrect variable assignment for SageMaker API call by @TristanHeisler in https://github.com/datahub-project/datahub/pull/10965
fix(airflow/build): Pinning mypy by @treff7es in https://github.com/datahub-project/datahub/pull/10972
Fixed a bug where the OpenAPI V3 spec was incorrect. The bug was introduced in https://github.com/datahub-project/datahub/pull/10939. by @ajoymajumdar in https://github.com/datahub-project/datahub/pull/10974
fix(ingest/test): Fix for mssql integration tests by @treff7es in https://github.com/datahub-project/datahub/pull/10978
fix(entity-service) exist check correctly extracts status by @jayacryl in https://github.com/datahub-project/datahub/pull/10973
fix(structuredProps) Fix casing bug in StructuredPropertiesValidator by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/10982
bugfix: use anyOf instead of allOf when creating references in openapi v3 spec by @kevin1chun in https://github.com/datahub-project/datahub/pull/10986
fix(ui): Remove ant less imports by @asikowitz in https://github.com/datahub-project/datahub/pull/10988
feat(ingest/graph): Add get_results_by_filter to DataHubGraph by @asikowitz in https://github.com/datahub-project/datahub/pull/10987
feat(ingest/cli): init does not actually support environment variables by @darnaut in https://github.com/datahub-project/datahub/pull/10989
fix(ingest/graph): Update get_results_by_filter graphql query by @asikowitz in https://github.com/datahub-project/datahub/pull/10991
feat(ingest/spark): Promote beta plugin by @treff7es in https://github.com/datahub-project/datahub/pull/10881
feat(ingest): support domains in meta -> "datahub" section by @hsheth2 in https://github.com/datahub-project/datahub/pull/10967
feat(ingest): add check server-config command by @hsheth2 in https://github.com/datahub-project/datahub/pull/10990
feat(cli): Make consistent use of DataHubGraphClientConfig by @pedro93 in https://github.com/datahub-project/datahub/pull/10466
fix(ingest/s3): Fixing container creation when there is no folder in path by @treff7es in https://github.com/datahub-project/datahub/pull/10993
fix(ingest/looker): support platform instance for dashboards & charts by @sid-acryl in https://github.com/datahub-project/datahub/pull/10771
feat(ingest/bigquery): improve handling of information schema in sql parser by @hsheth2 in https://github.com/datahub-project/datahub/pull/10985
feat(ingest): improve ingest deploy command by @hsheth2 in https://github.com/datahub-project/datahub/pull/10944
fix(backend): allow excluding soft-deleted entities in relationship-queries; exclude soft-deleted members of groups by @ksrinath in https://github.com/datahub-project/datahub/pull/10920
fix(ingest/looker): downgrade missing chart type log level by @hsheth2 in https://github.com/datahub-project/datahub/pull/10996
doc(acryl-cloud): release docs for 0.3.4.x by @anshbansal in https://github.com/datahub-project/datahub/pull/10984
fix(protobuf/build): Fix protobuf check jar script by @treff7es in https://github.com/datahub-project/datahub/pull/11006
fix(ui/ingest): Support invalid cron jobs by @asikowitz in https://github.com/datahub-project/datahub/pull/10998
fix(ingest): fix graph config loading by @hsheth2 in https://github.com/datahub-project/datahub/pull/11002
feat(docs): Document _DATAHUB_TO_FILE directive by @pedro93 in https://github.com/datahub-project/datahub/pull/10968
fix(graphql/upsertIngestionSource): Validate cron schedule; parse error in CLI by @asikowitz in https://github.com/datahub-project/datahub/pull/11011
feat(ece): support custom ownership type urns in ECE generation by @hsheth2 in https://github.com/datahub-project/datahub/pull/10999
feat(assertion-v2): changed Validation tab to Quality and created new Governance tab by @amit-apptware in https://github.com/datahub-project/datahub/pull/10935
fix(ingestion/glue): Add support for missing config options for profiling in Glue by @sagar-salvi-apptware in https://github.com/datahub-project/datahub/pull/10858
feat(propagation): Add models for schema field docs, tags, terms (#2959) by @shirshanka in https://github.com/datahub-project/datahub/pull/11016
docs: standardize terminology to DataHub Cloud by @yoonhyejin in https://github.com/datahub-project/datahub/pull/11003
fix(ingestion/transformer): tranformer to replace the externalUrl in container properties by @sagar-salvi-apptware in https://github.com/datahub-project/datahub/pull/11013
docs(slack) troubleshoot docs by @jayacryl in https://github.com/datahub-project/datahub/pull/11014
feat(propagation): Add graphql API by @shirshanka in https://github.com/datahub-project/datahub/pull/11030
feat(propagation): Add models for Action feature settings by @samblackk in https://github.com/datahub-project/datahub/pull/11029
docs(custom properties): Remove duplicate from sidebar by @eboneil in https://github.com/datahub-project/datahub/pull/11033
feat(models): Introducing Dataset Partitions Aspect by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/10997
feat(propagation): Add Documentation Propagation Settings by @samblackk in https://github.com/datahub-project/datahub/pull/11038
fix(models): chart schema fields mapping, add dataHubAction entity, t… by @shirshanka in https://github.com/datahub-project/datahub/pull/11040
fix(ci): smoke test lint failures by @shirshanka in https://github.com/datahub-project/datahub/pull/11044
docs: fix learning center color scheme & typo by @yoonhyejin in https://github.com/datahub-project/datahub/pull/11043
feat: add cloud main page by @yoonhyejin in https://github.com/datahub-project/datahub/pull/11017
feat(restore-indices): add additional step to also clear system metadata service by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10662
docs: fix typo by @yoonhyejin in https://github.com/datahub-project/datahub/pull/11046
fix(lint): apply spotless by @anshbansal in https://github.com/datahub-project/datahub/pull/11050
docs(airflow) Example query to get datajobs for a dataflow by @eboneil in https://github.com/datahub-project/datahub/pull/11034
feat(cli): Add run-id option to put sub-command by @pedro93 in https://github.com/datahub-project/datahub/pull/11023
fix(ingest): improve sql error reporting calls by @hsheth2 in https://github.com/datahub-project/datahub/pull/11025
fix(airflow): fix CI setup by @hsheth2 in https://github.com/datahub-project/datahub/pull/11031
feat(ingest/dbt): add experimental prefer_sql_parser_lineage flag by @hsheth2 in https://github.com/datahub-project/datahub/pull/11039
fix(ingestion/lookml): enable stack-trace in lookml logs by @sid-acryl in https://github.com/datahub-project/datahub/pull/10971
(chore): Linting fix by @rtekal in https://github.com/datahub-project/datahub/pull/11015
chore(ci): update deprecated github actions by @anshbansal in https://github.com/datahub-project/datahub/pull/10977
Fix ALB configuration example by @steffengr in https://github.com/datahub-project/datahub/pull/10981
chore(ingestion-base): bump base image packages by @david-leifker in https://github.com/datahub-project/datahub/pull/11053
feat(cli): Trim report of dataHubExecutionRequestResult to max GMS size by @pedro93 in https://github.com/datahub-project/datahub/pull/11051
fix(ingestion/lookml): emit dummy sql condition for lookml custom condition tag by @sid-acryl in https://github.com/datahub-project/datahub/pull/11008
fix(ingestion/powerbi): fix issue with broken report lineage by @sid-acryl in https://github.com/datahub-project/datahub/pull/10910
feat(ingest/tableau): add retry on timeout by @hsheth2 in https://github.com/datahub-project/datahub/pull/10995
change generate kafka connect properties from env by @wornjs in https://github.com/datahub-project/datahub/pull/10545
fix(ingest): fix oracle cronjob ingestion by @lima-renan in https://github.com/datahub-project/datahub/pull/11001
chore(ci): revert update deprecated github actions (#10977) by @david-leifker in https://github.com/datahub-project/datahub/pull/11062
feat(ingest/dbt-cloud): update metadata_endpoint inference by @hsheth2 in https://github.com/datahub-project/datahub/pull/11041
build: Reduce size of datahub-frontend-react image by 50-ish% by @xkollar in https://github.com/datahub-project/datahub/pull/10878
fix(ci): Fix lint issue in datahub_ingestion_run_summary_provider.py by @pedro93 in https://github.com/datahub-project/datahub/pull/11063
docs(ingest): update developing-a-transformer.md by @acrylJonny in https://github.com/datahub-project/datahub/pull/11019
feat(search-test): update search tests from #10408 by @david-leifker in https://github.com/datahub-project/datahub/pull/11056
feat(cli): add aspects parameter to DataHubGraph.get_entity_semityped by @Masterchen09 in https://github.com/datahub-project/datahub/pull/11009
docs(airflow): update min version for plugin v2 by @hsheth2 in https://github.com/datahub-project/datahub/pull/11065
doc(ingestion/tableau): doc update for derived permission by @sid-acryl in https://github.com/datahub-project/datahub/pull/11054
fix(py): remove dep on types-pkg_resources by @hsheth2 in https://github.com/datahub-project/datahub/pull/11076
feat(ingest/mode): add option to exclude restricted by @anshbansal in https://github.com/datahub-project/datahub/pull/11081
fix(ingest): set lastObserved in sdk when unset by @hsheth2 in https://github.com/datahub-project/datahub/pull/11071
doc(ingest): Update capabilities by @treff7es in https://github.com/datahub-project/datahub/pull/11072
chore(vulnerability): Log Injection by @pinakipb2 in https://github.com/datahub-project/datahub/pull/11090
chore(vulnerability): Information exposure through a stack trace by @pinakipb2 in https://github.com/datahub-project/datahub/pull/11091
chore(vulnerability): Comparison of narrow type with wide type in loop condition by @pinakipb2 in https://github.com/datahub-project/datahub/pull/11089
chore(vulnerability): Insertion of sensitive information into log files by @pinakipb2 in https://github.com/datahub-project/datahub/pull/11088
chore(vulnerability): Risky Cryptographic Algorithm by @pinakipb2 in https://github.com/datahub-project/datahub/pull/11059
chore(vulnerability): Overly permissive regex range by @pinakipb2 in https://github.com/datahub-project/datahub/pull/11061
fix: update customer data by @yoonhyejin in https://github.com/datahub-project/datahub/pull/11075
fix(models): fixing the datasetPartition models by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/11085
fix(ui): Adding view, forms GraphQL query, remove showing a fallback error message on unhandled GraphQL error by @jjoyce0510 in https://github.com/datahub-project/datahub/pull/11084
feat(docs-site) hiding learn more from cloud page by @jayacryl in https://github.com/datahub-project/datahub/pull/11097
fix(docs): Add correct usage of orFilters in search API docs by @gabe-lyons in https://github.com/datahub-project/datahub/pull/11082
fix(ingest/mode): Regexp in mode name matcher didn't allow underscore by @treff7es in https://github.com/datahub-project/datahub/pull/11098
docs: Refactor customer stories section by @yoonhyejin in https://github.com/datahub-project/datahub/pull/10869
fix(release): fix full/slim suffix on tag by @david-leifker in https://github.com/datahub-project/datahub/pull/11087
feat(config): support alternate hashing algorithm for doc id by @pinakipb2 in https://github.com/datahub-project/datahub/pull/10423
fix(emitter): fix typo in get method of java kafka emitter by @rtekal in https://github.com/datahub-project/datahub/pull/11007
fix(ingest): use correct native data type in all SQLAlchemy sources by compiling data type using dialect by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10898
chore: Update contributors list in PR labeler by @skrydal in https://github.com/datahub-project/datahub/pull/11105
feat(ingest): tweak stale entity removal messaging by @hsheth2 in https://github.com/datahub-project/datahub/pull/11064
fix(ingestion): enforce lastObserved timestamps in SystemMetadata by @david-leifker in https://github.com/datahub-project/datahub/pull/11104
fix(ingest/powerbi): fix broken lineage between chart and dataset by @sid-acryl in https://github.com/datahub-project/datahub/pull/11080
feat(ingest/lookml): CLL support for sql set in sql_table_name attribute of lookml view by @sid-acryl in https://github.com/datahub-project/datahub/pull/11069
docs: update graphql docs on forms & structured properties by @yoonhyejin in https://github.com/datahub-project/datahub/pull/11100
test(openAPI v3): search openAPI test has been added by @Kunal-kankriya in https://github.com/datahub-project/datahub/pull/11049
fix(ingest/tableau): prevent empty site content urls by @hsheth2 in https://github.com/datahub-project/datahub/pull/11057
feat(entity-client): implement client batch interface by @david-leifker in https://github.com/datahub-project/datahub/pull/11106
fix(snowflake): avoid reporting warnings/info for sys tables by @hsheth2 in https://github.com/datahub-project/datahub/pull/11114
fix(ingest): downgrade column type mapping warning to info by @hsheth2 in https://github.com/datahub-project/datahub/pull/11115
feat(api): add AuditStamp to the V3 API entity/aspect response by @ajoymajumdar in https://github.com/datahub-project/datahub/pull/11118
fix(ingest/redshift): replace r'\n' with '\n' to avoid token error redshift serverless… by @AndreasHegerNuritas in https://github.com/datahub-project/datahub/pull/11111
fix(entiy-client): handle null entityUrn case for restli by @david-leifker in https://github.com/datahub-project/datahub/pull/11122
fix(sql-parser): prevent bad urns from alter table lineage by @hsheth2 in https://github.com/datahub-project/datahub/pull/11092
fix(ingest/bigquery): use small batch size if use_tables_list_query_v2 is set by @mayurinehate in https://github.com/datahub-project/datahub/pull/11121
fix(graphql): add missing entities to EntityTypeMapper and EntityTypeUrnMapper by @Masterchen09 in https://github.com/datahub-project/datahub/pull/10366
Changes to allow editable dataset name by @jayasimhankv in https://github.com/datahub-project/datahub/pull/10608
fix: remove saxo by @yoonhyejin in https://github.com/datahub-project/datahub/pull/11127
feat(mcl-processor): Update mcl processor hooks by @david-leifker in https://github.com/datahub-project/datahub/pull/11134
docs(policies): updates to policies documentation by @david-leifker in https://github.com/datahub-project/datahub/pull/11073
fix(openapi): fix openapi v2 and v3 docs update by @david-leifker in https://github.com/datahub-project/datahub/pull/11139
feat(auth): grant type and acr values custom oidc parameters support by @RyanHolstien in https://github.com/datahub-project/datahub/pull/11116
fix(mutator): mutator hook fixes by @RyanHolstien in https://github.com/datahub-project/datahub/pull/11140
feat(search): support sorting on multiple fields by @RyanHolstien in https://github.com/datahub-project/datahub/pull/10775
feat(ingest): various logging improvements by @hsheth2 in https://github.com/datahub-project/datahub/pull/11126
fix(ingestion/lookml): fix for sql parsing error by @sid-acryl in https://github.com/datahub-project/datahub/pull/11079
feat(docs-site) cloud page spacing and content polishes by @jayacryl in https://github.com/datahub-project/datahub/pull/11141
feat(ui) Enable editing structured props on fields by @chriscollins3456 in https://github.com/datahub-project/datahub/pull/11042
feat(tests): add md5 and last computed to testResult model by @RyanHolstien in https://github.com/datahub-project/datahub/pull/11117
test(openapi): openapi regression smoke tests by @david-leifker in https://github.com/datahub-project/datahub/pull/11143
fix(airflow): fix tox tests + update docs by @hsheth2 in https://github.com/datahub-project/datahub/pull/11125

New Contributors

@obaltian made their first contribution in https://github.com/datahub-project/datahub/pull/10539
@pulsar256 made their first contribution in https://github.com/datahub-project/datahub/pull/10381
@john-claro-cko made their first contribution in https://github.com/datahub-project/datahub/pull/10258
@nephtyws made their first contribution in https://github.com/datahub-project/datahub/pull/10634
@aabharti-visa made their first contribution in https://github.com/datahub-project/datahub/pull/10612
@pie1nthesky made their first contribution in https://github.com/datahub-project/datahub/pull/10735
@amit-apptware made their first contribution in https://github.com/datahub-project/datahub/pull/10625
@PeamThom made their first contribution in https://github.com/datahub-project/datahub/pull/10496
@tkdrahn made their first contribution in https://github.com/datahub-project/datahub/pull/9433
@ipolding-cais made their first contribution in https://github.com/datahub-project/datahub/pull/10717
@brbrown25 made their first contribution in https://github.com/datahub-project/datahub/pull/10533
@simaov made their first contribution in https://github.com/datahub-project/datahub/pull/8415
@jordanjeremy made their first contribution in https://github.com/datahub-project/datahub/pull/10818
@aviv-julienjehannet made their first contribution in https://github.com/datahub-project/datahub/pull/10784
@chardaway made their first contribution in https://github.com/datahub-project/datahub/pull/9893
@shtephlee made their first contribution in https://github.com/datahub-project/datahub/pull/10909
@dragontail made their first contribution in https://github.com/datahub-project/datahub/pull/10877
@nadavgross made their first contribution in https://github.com/datahub-project/datahub/pull/10830
@joelmataKPN made their first contribution in https://github.com/datahub-project/datahub/pull/10813
@TristanHeisler made their first contribution in https://github.com/datahub-project/datahub/pull/10965
@samblackk made their first contribution in https://github.com/datahub-project/datahub/pull/11029
@steffengr made their first contribution in https://github.com/datahub-project/datahub/pull/10981
@wornjs made their first contribution in https://github.com/datahub-project/datahub/pull/10545
@lima-renan made their first contribution in https://github.com/datahub-project/datahub/pull/11001
@xkollar made their first contribution in https://github.com/datahub-project/datahub/pull/10878
@acrylJonny made their first contribution in https://github.com/datahub-project/datahub/pull/11019
@AndreasHegerNuritas made their first contribution in https://github.com/datahub-project/datahub/pull/11111

Full Changelog: https://github.com/datahub-project/datahub/compare/v0.13.3...v0.14.0

v0.13.3

Released on 2024-05-23 by @david-leifker.

DataHub Release Notes

User Experience

NEW: Business Attributes: Business Attributes are used to standardize and manage data elements across multiple domains, projects, and applications. By linking dataset attributes to Business Attributes, organizations ensure uniformity and ease of updates, as changes made to a Business Attribute are automatically propagated across all linked datasets. #9863
Improved UI for Dataset Properties: Added collapse functionality for long dataset properties, making it easier to navigate and view relevant information. [#10203](https://github.com/datahub-project/datahub/pull/10203)
Pagination for Ingestion Tasks Listing: Added pagination to the tasks listing page, making it easier to manage and navigate through tasks. [#10293](https://github.com/datahub-project/datahub/pull/10293)
Rich Text Support for Form Descriptions: Added support for rich text in form descriptions, enhancing the user experience. [#104](https://github.com/datahub-project/datahub/pull/104)2 5
New Analytics Charts: Added charts in the Analytics tab to identify Top Users and New Users. #10344
Enhanced search functionality with customizable autocomplete configuration. [#10426](https://github.com/datahub-project/datahub/pull/10426)

Developer Experience

Unified CI Workflow Updates: Improved CI build with unified workflow updates and disk space cleanup, making the build process more efficient. [#10353](https://github.com/datahub-project/datahub/pull/10353)
Improved Logging for GraphQL Requests: Enhanced logging for GraphQL requests, providing better insights and debugging capabilities. [#10404](https://github.com/datahub-project/datahub/pull/10404)
Enhanced Documentation for Lineage Feature Guide: Updated documentation for the lineage feature guide, making it easier to understand and implement. [#10401](https://github.com/datahub-project/datahub/pull/10401)
Improved Documentation for SchemaField.label: Updated documentation for SchemaField.label, providing clearer guidance for developers. [#10251](https://github.com/datahub-project/datahub/pull/10251)
Enhanced CI with Docker Image Publishing: Added Docker image publishing capabilities to the CI workflow, streamlining the deployment process. [#10193](https://github.com/datahub-project/datahub/pull/10193)
Redesigned Docs Site Feedback Button: Improved the design of the feedback button in the documentation, making it more user-friendly. [#10182](https://github.com/datahub-project/datahub/pull/10182)

Metadata Ingestion

Improved Data Profiling by early filtering of tables, correctly computing sample row counts, and combining unique count queries per table. #10378, #10319, #10322
Airflow: Introduced support for BigQueryInsertJobOperator. #10452
BigQuery: Added support for Table Clones and incremental column-level lineage.
Snowflake: Improved reporting for usage aggregation and handled lineage errors; Improved ingestion performance with system sampling on very large tables. #10279, #10430
Glue: Introduced support for delta schemas. [#10299](https://github.com/datahub-project/datahub/pull/10299)
Redshift: Improved usage extraction by filtering out system queries. #10247
Mode: Enhanced ingestion for Mode by adding dashboards into containers, improving data visualization and management. [#10563](https://github.com/datahub-project/datahub/pull/10563)
PowerBI: Added support to automatically extract table lineage between PowerBI and Databricks. [#10416](https://github.com/datahub-project/datahub/pull/10416)
dbt: Improved dbt ingestion by handling complex SQL and enhancing documentation, providing better data management and insights. [#10323](https://github.com/datahub-project/datahub/pull/10323)
NiFi: Enhanced ingestion for NiFi with process group as browse path and incremental lineage, improving data organization and tracking. [#10202](https://github.com/datahub-project/datahub/pull/10202)
Incubating Sigma and CockroachDB sources. #10037, #10226

Breaking Changes

DynamoDB Connector: aws_region is now a required configuration. The connector will no longer loop through all AWS regions; instead, it will only use the region passed into the recipe configuration. [#10419](https://github.com/datahub-project/datahub/pull/10419)
Custom Validators and Mutators: Dropped a previously required constructor. [#10389](https://github.com/datahub-project/datahub/pull/10389)
FabricType RVW: Added as a new FabricType. No rollbacks allowed once metadata with this fabric type is added without manual cleanups in databases. [#10472](https://github.com/datahub-project/datahub/pull/10472)

For full details on breaking changes, please refer to the updating DataHub documentation.