Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sdk): NewMetaWrapper initializes the epoch with a random number. #3417

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

yuangcao
Copy link

close: #3416

Fixes: #3416

Motivation

I use the mdtest with CubeFS backend support to measure the throughput of CubeFS meta subsystem. Each mdtest process will create its own CubeFS client with the MetaWrapper having the same epoch.

Each mdtest process creates the same directory tree (one dir contains millions of files). In this situation, all mdtest processes will create their root dir inode in the same meta partition, because all the mdtest processes' CubeFS client have the same epoch. Then all the files' dentry will be created in the same meta partition, which will cause significant performance degradation.

Modifications

In the NewMetaWrapper function, I assign a random value to the mw.epoch. The value is in the range of [0, len(mw.rwPartitions)].

Types of changes

  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Bugfix (non-breaking change which fixes an issue)
  • Documentation Update (if none of the other choices apply)
  • So on...

Verifying this change

  • Make sure that the change passes the testing checks.

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • This can be verified in development debugging
  • This can be realized in a mocked environment, like a test cluster consisting in docker

(or)

This change MUST reappear in online clusters, or occur in that specific scenarios.

Does this pull request potentially affect one of the following parts:

  • Master
  • MetaNode
  • DataNode
  • ObjectNode
  • AuthNode
  • LcNode
  • Blobstore
  • Client
  • Cli
  • SDK
  • Other Tools
  • Common Packages
  • Dependencies
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Review Expection

  • in-two-days
  • weekly
  • free-time
  • whenever

Copy link

netlify bot commented Jun 19, 2024

Deploy Preview for cubefs-check ready!

Name Link
🔨 Latest commit 2682ab6
🔍 Latest deploy log https://app.netlify.com/sites/cubefs-check/deploys/6672dc740bbda100088aad9d
😎 Deploy Preview https://deploy-preview-3417--cubefs-check.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@yuangcao yuangcao changed the title enhance(sdk): NewMetaWrapper initializes the epoch with a random number. fix(sdk): NewMetaWrapper initializes the epoch with a random number. Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement]: meta.NewMetaWrapper should initialize the epoch with a random number
1 participant