Re: unversioned properties: size limitations?

On 12.08.2014 07:16, Alexey Neyman wrote:
>
> On Tuesday, August 12, 2014 06:59:20 AM Branko Čibej wrote:
>
> > On 12.08.2014 03:31, Alexey Neyman wrote:
>
> > > Hi SVN users/developers,
>
> > >
>
> > > Is there a limitation in size on the property value that can be set?
>
> > > Any scalability traps to be aware of (i.e. non-linear increase in time
>
> > > due to increase in size of the property value)? I tried a 4Mb
>
> > > property, seems to work fine...
>
> >
>
> > One thing to be aware of is that properties were never designed to be
>
> > large. Property values are always transmitted in full text between
>
> > client and server (i.e., they're not compressed); they're stored in full
>
> > text in the repository (not deltified the way file contents are). So the
>
> > more large properties you have, and the more often you modify them, the
>
> > less efficient your repository will be, in terms of storage requirements
>
> > and network bandwidth.
>
> >
>
> > So while you should be able to store a 2 gig property value, I really,
>
> > really recommend not to do that.
>
>
>
> I thought of having pre- and post-commit hooks communicate using a
> *revision* property: pre-commit hook would set a revision property
> with the list of files and actions to be performed on them, and the
> post-commit hook will perform these actions by committing a new
> revision (instead of modifying a transaction by pre-commit hook).
>
>
>
> Thus a more specific question - when are arbitrary *revision*
> properties sent from the client to the server? Obviously, svn:*
> properties are used by various SVN commands; but am I right to assume
> that non-standard revision properties are sent only for the 'svn pg
> --revprop' command?
>
>
>
> That said, I expect the property value to be much less than 2Gb. So
> far, the largest commit we've had was ~20000 files - with ~150
> characters per path, that would be about 3Mb for the property value.
>

Sure, you'll only transmit revprops with propget --revprop and propset
--revprop. I'm not sure what the implications are of storing large
values in revprops, these are handled a bit differently than versioned
properties on the server.

And of course, revision properties are not versioned.

I'm still not sure what you're trying to achieve, though. "Communication
between pre- and post-commit hooks" doesn't describe the problem, it
describes a solution, and there are of course other ways for hooks to
communicate that do not involve the repository.

Also I find your approach less than robust:

  * There's no guarantee that the post-commit hook will ever run, so
    it's a bad idea to rely on it for anything that's critical to your
    workflow.
  * There's no guarantee that other commits won't happen before your
    post-commit hook is run; so whatever you do with the repository in
    post-commit may have to deal with conflicts, which is not fun to
    automate.

Why don't you just describe what kind of workflow you're implementing,
and more importanly, why?

-- Brane

-- 
Branko Čibej | Director of Subversion
WANdisco | Realising the impossibilities of Big Data
e. brane_at_wandisco.com