Re: [PATCH] Fix "database is locked" error when committing during rep-cache.db verification

> > > 1. a post-commit error "database is locked"
> > > 2. new representations will not be added in the rep-cache.db
> > > 3. deduplication does not work for new data committed at this time
> > > 4. commits work with delays.

> > I look to hearing Denis's concerns with the sharding approach.
>
> In addition, if we ever need atomic operations on the entire
> rep-cache, we will have to use ATTACH DATABASE statement [1] with the master
> journal [2], which I think is not used anywhere now, and is not supported by
> all journaling modes.
>

Fair point. Let's see the pros and cons:

- pro: fixes issues #1 through #4, and the "large commits starve the following
  commit" issue as well

- con: if someday we have a need for cross-shard atomic transactions, we'll
  have to either (1) forbid rep-cache.db shards from using WAL mode; or
  (2) figure out another way to solve whichever problem we would use atomic
  cross-shard transactions to solve if we had an unsharded rep-cache.db; or
  (3) revert to a monolithic rep-cache.db and figure out another way to solve
  the problems this thread is about.

> - If I'm not mistaken, this approach requires a format bump. So this does not
> fix the problem for existing repositories. It is also necessary to perform
> the division into shards in some form, which means that a fast in-place
> upgrade also probably will not fix the problem.

An in-place upgrade would be straightforward.

>

I've snipped most of your points as I already answered them upthread. If
I didn't respond to some point which _hasn't_ already been addressed upthread,
just ask it again.

My concerns stand.

Daniel
Received on 2020-05-20 05:32:52 CEST