> > > 1. a post-commit error "database is locked"
> > > 2. new representations will not be added in the rep-cache.db
> > > 3. deduplication does not work for new data committed at this time
> > > 4. commits work with delays.
> > I look to hearing Denis's concerns with the sharding approach.
>
> In addition, if we ever need atomic operations on the entire
> rep-cache, we will have to use ATTACH DATABASE statement [1] with the master
> journal [2], which I think is not used anywhere now, and is not supported by
> all journaling modes.
>
Fair point. Let's see the pros and cons:
- pro: fixes issues #1 through #4, and the "large commits starve the following
commit" issue as well
- con: if someday we have a need for cross-shard atomic transactions, we'll
have to either (1) forbid rep-cache.db shards from using WAL mode; or
(2) figure out another way to solve whichever problem we would use atomic
cross-shard transactions to solve if we had an unsharded rep-cache.db; or
(3) revert to a monolithic rep-cache.db and figure out another way to solve
the problems this thread is about.
> - If I'm not mistaken, this approach requires a format bump. So this does not
> fix the problem for existing repositories. It is also necessary to perform
> the division into shards in some form, which means that a fast in-place
> upgrade also probably will not fix the problem.
An in-place upgrade would be straightforward.
>
I've snipped most of your points as I already answered them upthread. If
I didn't respond to some point which _hasn't_ already been addressed upthread,
just ask it again.
My concerns stand.
Daniel
Received on 2020-05-20 05:32:52 CEST