Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Refine quickstart docs further for 1.0 #12420

Merged
merged 16 commits into from
Dec 7, 2024

Conversation

lokeshj1703
Copy link
Contributor

@lokeshj1703 lokeshj1703 commented Dec 4, 2024

Change Logs

Adds more quickstart docs around indexing, partial updates and merge mode.
Further refines the indexing docs, adds limitations and index acceleration related information.

Impact

Refines quickstart docs

Risk level (write none, low medium or high below)

low

Documentation Update

NA

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@lokeshj1703 lokeshj1703 changed the base branch from master to asf-site December 4, 2024 09:10
@github-actions github-actions bot added docs size:M PR with lines of changes in (100, 300] labels Dec 4, 2024
website/docs/metadata_indexing.md Outdated Show resolved Hide resolved
website/docs/sql_ddl.md Outdated Show resolved Hide resolved
@github-actions github-actions bot added size:L PR with lines of changes in (300, 1000] and removed size:M PR with lines of changes in (100, 300] labels Dec 4, 2024
website/docs/metadata.md Outdated Show resolved Hide resolved
Copy link
Member

@vinothchandar vinothchandar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, can you please confirm if we can trim down some extra configs (which are not needed due to changed defaults) and how we use record mergers and not payloads.

website/docs/metadata_indexing.md Outdated Show resolved Hide resolved
MERGE INTO hudi_table AS target
USING fare_adjustment AS source
ON target.uuid = source.uuid
WHEN MATCHED THEN UPDATE SET fare = source.fare
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, what exactly is the modification from example above. the previous one does not use UPDATE SET *

website/docs/quick-start-guide.md Outdated Show resolved Hide resolved
website/docs/quick-start-guide.md Outdated Show resolved Hide resolved
website/docs/quick-start-guide.md Outdated Show resolved Hide resolved
website/docs/sql_ddl.md Show resolved Hide resolved
primaryKey = 'id',
precombineField = 'ts',
recordMergeMode = 'CUSTOM',
'hoodie.datasource.write.payload.class' = 'org.apache.hudi.common.model.PartialUpdateAvroPayload'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need to show merger... not a payload.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok i will update with the merger example.

website/docs/sql_ddl.md Show resolved Hide resolved
website/docs/sql_ddl.md Outdated Show resolved Hide resolved
@lokeshj1703
Copy link
Contributor Author

Overall, can you please confirm if we can trim down some extra configs (which are not needed due to changed defaults) and how we use record mergers and not payloads.

Addressed both. Removed hoodie.enable.data.skipping=true usage and hoodie.metadata.enable=true usage. Also using record merge mode instead of payload now for secondary index.

Copy link
Member

@vinothchandar vinothchandar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll make another pass

@vinothchandar vinothchandar merged commit a0bf38e into apache:asf-site Dec 7, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs size:L PR with lines of changes in (300, 1000]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants