[PB-6472]: feat/track-user-mail-usage#220
Conversation
jzunigax2
changed the title
|
The This makes the network less coupled to mail/drive/any product we may release in the future that consumes space while gaining standarisation of the calculus of the used space on a bucket. WDYT @apsantiso @jzunigax2 ? This may be even useful for Photos or the same Drive in any foreseeable future. |
jzunigax2
force-pushed
the
feat/track-user-mail-usage
branch
from
02b5882 to
6f17e6b
Compare
Add a usedSpaceBytes field to buckets and a gateway endpoint to set it, so products whose content lives outside the network (mail/Stalwart) can report the space they consume against a network bucket.
Add a new method to sum used space bytes in MongoDBBucketsRepository and update the UsersUsecase to return a UserSpaceSnapshot when setting bucket usage. Update HTTPGatewayController to send the snapshot in the response. Include tests to verify the new functionality.
Add methods to create and delete bucket entries in the MongoDBBucketEntriesRepository and BucketEntriesUsecase. Introduce a new endpoint in the HTTPGatewayController for creating entries, which includes validation for entry keys and sizes. Update tests to cover the new functionality
…cketEntriesUsecase and its associated tests
jzunigax2
force-pushed
the
feat/track-user-mail-usage
branch
from
3ab3f05 to
c8aa165
Compare
| maxSpaceBytes: user.maxSpaceBytes, | ||
| totalUsedSpaceBytes: created | ||
| ? user.totalUsedSpaceBytes + size | ||
| : user.totalUsedSpaceBytes, |
There was a problem hiding this comment.
It's possible to update the size? Because in that case, you should consider the sum of it to the total. If this is not possible, close the comment. I would personally get what the database returns regarding the user totalUsedSpaceBytes field — if possible — and use the value directly to make this far simpler
There was a problem hiding this comment.
right, now using what's returned by findOneAndUpdate
| const user = await this.findUserAndOwnedBucket(userUuid, bucketId); | ||
|
|
||
| const index = this.hashEntryKey(key); | ||
| const entry = await this.bucketEntriesRepository.findOne({ bucket: bucketId, index }); |
There was a problem hiding this comment.
Why are u using the index? It's any kind of workaround to relate the mail blob with the bucketEntry?
There was a problem hiding this comment.
yes its a sort of workaround, mail blobs have neither shards nor an encryption index but these were still required. index stores sha256(key) which allows lookups
There was a problem hiding this comment.
The index is not a field for this, sincerely. How drive does it is by holding the reference to the network file, rather than coupling the network to a specific service. Mail should be the one holding the pointer to here, not the other way around
… addTotalUsedSpaceBytes to return total used space
2 participants
What
Adds per-message storage accounting for Mail into the network, so a user's totalUsedSpaceBytes reflects Drive + Mail as a single real total. Mail writes one shard-less, version-2 BucketEntry per ingested message (keyed by accountId:documentId) and increments totalUsedSpaceBytes on first insert; deletes remove the entry and decrement.
Mail provisions one bucket per mail account #221 and periodically reports that account's usage from Stalwart here.
Why
Bridge needs to track each user's mail storage usage next to their Drive usage so the two can be checked against a single quota. This replaces the previous "report aggregate mail usage to a separate field" approach, which was rejected because that figure was not part of totalUsedSpaceBytes and therefore couldn't gate Drive uploads
The current design folds Mail into the same counter Drive already enforces against, using the same per-object metadata model as Drive's /finish.
Mail bodies and attachments are stored by Stalwart's own blob store and never traverse the network upload->shard->mirror path. There are therefore no real shards to record.
How
POST /v2/gateway/users/:uuid/buckets/:bucketId/entries