Performance of hashing on different file systems

Discussions
1

I also asked about this here: How much faster is turborepo on ext4 filesystems than APFS? · vercel/turborepo · Discussion #10014 · GitHub

I’m been noticing some very shocking things with APFS, that are honestly quite concerning.

Here is the test I setup: disk-perf-git-and-pnpm/README.md at main · NullVoxPopuli/disk-perf-git-and-pnpm · GitHub
along with a number of datapoints from various machines and operating systems.
Interactive table here

image
image1089×671 139 KB

This doesn’t test turbo specifically, but more file creation and deletion.

turborepo has to read all files in a package (governed by inputs), to use as part of what creates the hash for a task.

Now, I know read is different from create and delete, but still… I wonder.

Is there a test we can do with turbo in a sufficiently sized repo on both linux/ext4 and macOS/APFS to see how long it takes just to compute hashes for tasks?

As an aside, if anyone is curious about disk perf on APFS, I could really use your help if you have an M2, M3, or M4,

Instructions here: disk-perf-git-and-pnpm/README.md at main · NullVoxPopuli/disk-perf-git-and-pnpm · GitHub

tl;dr: time ( a git clean ) and time ( dep install )

thank you!!

3

Thanks for reaching out about your file system performance concerns! It looks like you’ve been talking with someone from the Turborepo team in your original GitHub discussion, so I’m going to close this topic as a duplicate.

Please feel free to open a new topic if you have any other questions!

1 Like