Discussion:
File Versioning for the KDE Desktop...
(too old to reply)
s***@osfda.org
2024-10-03 20:20:01 UTC
Permalink
I have a package that manages file versioning that has been alpha tested
with:

* bookworm

* KDE plasma

Think like: Windows File versioning/Mac Revert. It is based on inotify.
[Before you think that couldn't be reliable, hear me out for a
discussion of safeguards I take to make it so -at some future point I
could eventually do the heavy lift to implement it as a shim to the EXT4
driver., but for now: it works..]

While there are paid backup products that do this, there's no free
solution, ready to install from the stable repos (you can suggest any
that I might have missed, and I will confirm or show how they fall
short...) Presently Windows and Mac users at this point expect
integrated versioning to just be part of a desktop OS. With the rise in
Linux desktop adoption, having access to such a product that they have
come to rely on will help reduce the friction for some users migrating
to it.

I would like to get a sponsor for this package. I have a press article
for it, and it will have a dedicated vanity domain (purchased,
org+com...) I do not expect to have it included with KDE/plasma to
start, but it would greatly stimulate interest in it if it could be
readily accessible as a separate package, and it will also facilitate
getting people to field-test it out even further (for later adoption as
an "extra" in the KDE install?)

This package could eventually be adapted to gnome and other desktops,
but to start I have focused on KDE/plasma (it has integration with the
"Dolphin" file manager, though command-line operation is eminently
usable...)

The code base is written in python, and a service uses code generated by
cython at the time its package install is done (you can also opt to have
the service run using the python codebase, should more extended
traceback analysis ever be needed...)

I have assembled a prototype deb package file, and ran it through
linitian. I will have a few brief questions about standards for whoever
my sponsor ends up being.

If this plane takes off, I will put up a wiki for its manual on its
vanity domain, and its README and [pyqt5] GUI will have a link to it. I
could also opt to use Sphinx <https://www.sphinx-doc.org/en/master> for
its documentation, which I have used before and I readily admit has a
sharp look; but mediawiki <https://www.mediawiki.org/wiki/MediaWiki> can
be helpful in garnering the assistance of others with documentation at
some later point ("many hands make light work...")

I presently have a nice article with screenshots (as of yet unpublished,
pending its hopefully eventual appearance in bookworm's stable repo...)
*I would appreciate if someone from the KDE/plasma team sponsor this
package for inclusion to the stable repos at the first practical
opportunity; again, NOT for immediate inclusion into KDE/plasma.* After
months of field testing, perhaps it can */eventually/* be considered
worthy enough for KDE extras...

Respectfully,
Steve Boriotti
Senior Developer, full stack

------------------------------------------------------------------------

About me: I am a developer with over 30 years commercial experience
(mostly in fintech...) Back in the 2000s I did code auditing of the open
source Cyrus mail server <https://www.cyrusimap.org> (using "RATS
<https://code.google.com/archive/p/rough-auditing-tool-for-security>"...)
I also happen to be a postgres advocate, and have published a python
tool for generating self-signed certificates for pgadmin (a rather hairy
multistep process when done manually...)
[https://gitlab.com/osfda/pg_ssl_init] I even furnished patches on
StackOverflow to remedy bugs in the new asynchronous psycopg3
<https://www.psycopg.org/psycopg3> driver.
Antonio Russo
2024-10-04 02:20:01 UTC
Permalink
Post by s***@osfda.org
I have a package that manages file versioning that has been alpha tested
* bookworm
* KDE plasma
I apologize if I'm overlooking it, but could you please link to the source code
of this project, as well as your packaging of the deb?
Post by s***@osfda.org
Think like: Windows File versioning/Mac Revert. It is based on inotify. [Before
you think that couldn't be reliable, hear me out for a discussion of safeguards
I take to make it so -at some future point I could eventually do the heavy lift
to implement it as a shim to the EXT4 driver., but for now: it works..]
This sounds interesting!
Post by s***@osfda.org
While there are paid backup products that do this, there's no free solution,
ready to install from the stable repos (you can suggest any that I might have
missed, and I will confirm or show how they fall short...) Presently Windows
and Mac users at this point expect integrated versioning to just be part of a
desktop OS. With the rise in Linux desktop adoption, having access to such a
product that they have come to rely on will help reduce the friction for some
users migrating to it.
I disagree here; there are tools that do this. Git and ZFS and sanoid come to
mind, but these are not terribly new-user friendly.

There's also Nextcloud [1], (granted, the server is not packaged in stable).
Post by s***@osfda.org
I would like to get a sponsor for this package. I have a press article for it,
and it will have a dedicated vanity domain (purchased, org+com...) I do not
expect to have it included with KDE/plasma to start, but it would greatly
stimulate interest in it if it could be readily accessible as a separate
package, and it will also facilitate getting people to field-test it out even
further (for later adoption as an "extra" in the KDE install?)
This package could eventually be adapted to gnome and other desktops, but to
start I have focused on KDE/plasma (it has integration with the "Dolphin" file
manager, though command-line operation is eminently usable...)
The code base is written in python, and a service uses code generated by cython
at the time its package install is done (you can also opt to have the service
run using the python codebase, should more extended traceback analysis ever be
needed...)
I have assembled a prototype deb package file, and ran it through linitian. I
will have a few brief questions about standards for whoever my sponsor ends up
being.
If this plane takes off, I will put up a wiki for its manual on its vanity
domain, and its README and [pyqt5] GUI will have a link to it. I could also opt
to use Sphinx <https://www.sphinx-doc.org/en/master> for its documentation,
which I have used before and I readily admit has a sharp look; but mediawiki
<https://www.mediawiki.org/wiki/MediaWiki> can be helpful in garnering the
assistance of others with documentation at some later point ("many hands make
light work...")
I presently have a nice article with screenshots (as of yet unpublished,
pending its hopefully eventual appearance in bookworm's stable repo...) *I
would appreciate if someone from the KDE/plasma team sponsor this package for
inclusion to the stable repos at the first practical opportunity; again, NOT
for immediate inclusion into KDE/plasma.* After months of field testing,
perhaps it can */eventually/* be considered worthy enough for KDE extras...
I recommend reading [2] if you plan on maintaining this package yourself in
Debian.
Post by s***@osfda.org
Respectfully,
Steve Boriotti
Senior Developer, full stack
------------------------------------------------------------------------
About me: I am a developer with over 30 years commercial experience (mostly in
fintech...) Back in the 2000s I did code auditing of the open source Cyrus mail
server <https://www.cyrusimap.org> (using "RATS
<https://code.google.com/archive/p/rough-auditing-tool-for-security>"...) I
also happen to be a postgres advocate, and have published a python tool for
generating self-signed certificates for pgadmin (a rather hairy multistep
process when done manually...) [https://gitlab.com/osfda/pg_ssl_init] I even
furnished patches on StackOverflow to remedy bugs in the new asynchronous
psycopg3 <https://www.psycopg.org/psycopg3> driver.
Best,
Antonio

[1] https://docs.nextcloud.com/server/20/admin_manual/configuration_files/file_versioning.html
[2] https://www.debian.org/doc/manuals/maint-guide/
s***@osfda.org
2024-10-04 03:10:01 UTC
Permalink
Yes, well aware of git's potential to be used. But this is supersimple,
run by means of context menus in the dolphin file manager. It also uses
a server process to assign immutability to the archived files, to make
sure you don't */accidentally/* delete them (let's say you delete the
folder. being versioned...) The files are kept in a ".archive" subfolder
of the folder being versioned (so it's a simple local backup, just like
Windows File Versioning and Mac Revert...)

ZFS could work, but many will want to stay with the filesystem herd
(EXT4...)

I could eventually add a cloud storage capability, but that opens a can
of worms. Maybe */eventually/* just interface it to a git repo of one's
choosing? In the meantime, we have the equivalent functionality to File
Versioner/Revert.

I will need to add a "rules" and a few of these other files to it:
https://www.debian.org/doc/manuals/maint-guide/dother.en.html

A functional package and repo should be ready by Monday. I think one
should read the article first though before investing time in pouring
over code, to check that my approach is sound and does not violate any
security model.

Shoot me a non-list Email if you are interested, and I'll send the
article to you; it's about 10 pages and NOT technically dense (it's
geared more as a press kit for potential end-users...) Until it's
submitted, I'd rather not have another project grabbing my ideas and
later claiming that they came up with them first. [If you can't see my
Email on this list, I can convey it back by a trivial obfuscation,
unless you know another way; I could also DM you on Stack Overflow if
you have a profile there...]
------------------------------------------------------------------------
Sounds like you know what you're doing. So I have a couple of QUICK
questions on deb policy, if you wouldn't mind answering them. [I
searched for an answer, but the advice was a bit contested...]

* There's /usr/local/share, /usr/local/share/doc, and /opt for storing
package files. But a more sensible convention I have seen [for
non-standard packages, and this would certainly be that...] is to
store a package's associated components in a folder under /usr/local
(NOT /usr/local/*/share/*, because I don't want to suggest that all
of them be shared, since some are for a dedicated Linux service...)
Then for any commands, you set up symbolic links in /usr/local/bin
to scripts/binaries in that /usr/local subfolder (presently I have a
postinst that does that linking, long with a prerm to remove
them...) Does that sound OK to you? Because lintian was griefing
that I was putting a folder with files in /usr/local -hey, it IS
local, so in theory we should be able to adapt it as we see fit
(perhaps this is a case of lintian being a chatty linter...)

* KDE has a *requirement* that .desktop files be executable; lintian
saw that and certain group/other permissions as being weird. Can I
ignore those too? If not, I could always do the permissions the way
it wants, and adjust them after they are copied in, in the postinst
script...

Thanks again...
Post by Antonio Russo
Post by s***@osfda.org
I have a package that manages file versioning that has been alpha
 * bookworm
 * KDE plasma
I apologize if I'm overlooking it, but could you please link to the source code
of this project, as well as your packaging of the deb?
Post by s***@osfda.org
Think like: Windows File versioning/Mac Revert. It is based on
inotify. [Before you think that couldn't be reliable, hear me out for
a discussion of safeguards I take to make it so -at some future point
I could eventually do the heavy lift to implement it as a shim to the
EXT4 driver., but for now: it works..]
This sounds interesting!
Post by s***@osfda.org
While there are paid backup products that do this, there's no free
solution, ready to install from the stable repos (you can suggest any
that I might have missed, and I will confirm or show how they fall
short...) Presently Windows and Mac users at this point expect
integrated versioning to just be part of a desktop OS. With the rise
in Linux desktop adoption, having access to such a product that they
have come to rely on will help reduce the friction for some users
migrating to it.
I disagree here; there are tools that do this.  Git and ZFS and sanoid
come to
mind, but these are not terribly new-user friendly.
There's also Nextcloud [1], (granted, the server is not packaged in stable).
Post by s***@osfda.org
I would like to get a sponsor for this package. I have a press
article for it, and it will have a dedicated vanity domain
(purchased, org+com...) I do not expect to have it included with
KDE/plasma to start, but it would greatly stimulate interest in it if
it could be readily accessible as a separate package, and it will
also facilitate getting people to field-test it out even further (for
later adoption as an "extra" in the KDE install?)
This package could eventually be adapted to gnome and other desktops,
but to start I have focused on KDE/plasma (it has integration with
the "Dolphin" file manager, though command-line operation is
eminently usable...)
The code base is written in python, and a service uses code generated
by cython at the time its package install is done (you can also opt
to have the service run using the python codebase, should more
extended traceback analysis ever be needed...)
I have assembled a prototype deb package file, and ran it through
linitian. I will have a few brief questions about standards for
whoever my sponsor ends up being.
If this plane takes off, I will put up a wiki for its manual on its
vanity domain, and its README and [pyqt5] GUI will have a link to it.
I could also opt to use Sphinx <https://www.sphinx-doc.org/en/master>
for its documentation, which I have used before and I readily admit
has a sharp look; but mediawiki
<https://www.mediawiki.org/wiki/MediaWiki> can be helpful in
garnering the assistance of others with documentation at some later
point ("many hands make light work...")
I presently have a nice article with screenshots (as of yet
unpublished, pending its hopefully eventual appearance in bookworm's
stable repo...) *I would appreciate if someone from the KDE/plasma
team sponsor this package for inclusion to the stable repos at the
first practical opportunity; again, NOT for immediate inclusion into
KDE/plasma.* After months of field testing, perhaps it can
*/eventually/* be considered worthy enough for KDE extras...
I recommend reading [2] if you plan on maintaining this package yourself in
Debian.
Post by s***@osfda.org
Respectfully,
Steve Boriotti
Senior Developer, full stack
------------------------------------------------------------------------
About me: I am a developer with over 30 years commercial experience
(mostly in fintech...) Back in the 2000s I did code auditing of the
open source Cyrus mail server <https://www.cyrusimap.org> (using
"RATS
<https://code.google.com/archive/p/rough-auditing-tool-for-security>"...)
I also happen to be a postgres advocate, and have published a python
tool for generating self-signed certificates for pgadmin (a rather
hairy multistep process when done manually...)
[https://gitlab.com/osfda/pg_ssl_init] I even furnished patches on
StackOverflow to remedy bugs in the new asynchronous psycopg3
<https://www.psycopg.org/psycopg3> driver.
Best,
Antonio
[1]
https://docs.nextcloud.com/server/20/admin_manual/configuration_files/file_versioning.html
[2] https://www.debian.org/doc/manuals/maint-guide/
Andrey Rakhmatullin
2024-10-04 06:10:01 UTC
Permalink
Post by s***@osfda.org
Sounds like you know what you're doing. So I have a couple of QUICK
questions on deb policy, if you wouldn't mind answering them. [I searched
for an answer, but the advice was a bit contested...]
* There's /usr/local/share, /usr/local/share/doc, and /opt for storing
package files. But a more sensible convention I have seen [for
non-standard packages, and this would certainly be that...]
You said earlier that you want a sponsor, if by that you meant that you
want this package to be uploaded to Debian then it cannot use /usr/local
or /opt. Otherwise, the Policy doesn't apply to unofficial packages that
won't go into Debian so you can do whatever you want in those and it
doesn't make sense to fix all lintian output in them. debian-mentors@ OTOH
is not a correct place to discuss those.
Post by s***@osfda.org
* KDE has a *requirement* that .desktop files be executable
Does it?
--
WBR, wRAR
Sune Vuorela
2024-10-04 12:20:01 UTC
Permalink
Post by Andrey Rakhmatullin
Post by s***@osfda.org
* KDE has a *requirement* that .desktop files be executable
Does it?
No. But it is a requirement for .desktop files in nonstandard locations
for launching them to work.

(so that you don't download a .desktop file and double click it to
launch it)

/Sune
s***@osfda.org
2024-10-04 12:30:01 UTC
Permalink
Thanks; I thought it didn't work without executable when I had tried it,
but I guess I was mistaken...
Post by Sune Vuorela
Post by Andrey Rakhmatullin
Post by s***@osfda.org
* KDE has a *requirement* that .desktop files be executable
Does it?
No. But it is a requirement for .desktop files in nonstandard locations
for launching them to work.
(so that you don't download a .desktop file and double click it to
launch it)
/Sune
s***@osfda.org
2024-12-15 18:50:01 UTC
Permalink
Good day-

I finally got the documentation for my package squared away, along with
support for _both_ gitlab and github interoperability (with github
having a much more idiosyncratic REST API...)

* My package's documentation is here: https://repliversion.org

* My package's repo is here: https://git.repliversion.org

* The .deb for my package is here: https://deb.repliversion.org

The deb for my package works, but since it is the first one I have ever
assembled, there is a good chance it has deficiencies.

What are the next steps to getting this into unstable?

I think this package will be appreciated because EASY versioning is
something that's missing from desktop Linux, while competing desktop
OS's already have it (I believe the fact they do points to ts importance...)

[Note that the versioning works even in shell/non-desktop.]
Post by Antonio Russo
Post by s***@osfda.org
I have a package that manages file versioning that has been alpha
 * bookworm
 * KDE plasma
I apologize if I'm overlooking it, but could you please link to the source code
of this project, as well as your packaging of the deb?
Post by s***@osfda.org
Think like: Windows File versioning/Mac Revert. It is based on
inotify. [Before you think that couldn't be reliable, hear me out for
a discussion of safeguards I take to make it so -at some future point
I could eventually do the heavy lift to implement it as a shim to the
EXT4 driver., but for now: it works..]
This sounds interesting!
Post by s***@osfda.org
While there are paid backup products that do this, there's no free
solution, ready to install from the stable repos (you can suggest any
that I might have missed, and I will confirm or show how they fall
short...) Presently Windows and Mac users at this point expect
integrated versioning to just be part of a desktop OS. With the rise
in Linux desktop adoption, having access to such a product that they
have come to rely on will help reduce the friction for some users
migrating to it.
I disagree here; there are tools that do this.  Git and ZFS and sanoid
come to
mind, but these are not terribly new-user friendly.
There's also Nextcloud [1], (granted, the server is not packaged in stable).
Post by s***@osfda.org
I would like to get a sponsor for this package. I have a press
article for it, and it will have a dedicated vanity domain
(purchased, org+com...) I do not expect to have it included with
KDE/plasma to start, but it would greatly stimulate interest in it if
it could be readily accessible as a separate package, and it will
also facilitate getting people to field-test it out even further (for
later adoption as an "extra" in the KDE install?)
This package could eventually be adapted to gnome and other desktops,
but to start I have focused on KDE/plasma (it has integration with
the "Dolphin" file manager, though command-line operation is
eminently usable...)
The code base is written in python, and a service uses code generated
by cython at the time its package install is done (you can also opt
to have the service run using the python codebase, should more
extended traceback analysis ever be needed...)
I have assembled a prototype deb package file, and ran it through
linitian. I will have a few brief questions about standards for
whoever my sponsor ends up being.
If this plane takes off, I will put up a wiki for its manual on its
vanity domain, and its README and [pyqt5] GUI will have a link to it.
I could also opt to use Sphinx <https://www.sphinx-doc.org/en/master>
for its documentation, which I have used before and I readily admit
has a sharp look; but mediawiki
<https://www.mediawiki.org/wiki/MediaWiki> can be helpful in
garnering the assistance of others with documentation at some later
point ("many hands make light work...")
I presently have a nice article with screenshots (as of yet
unpublished, pending its hopefully eventual appearance in bookworm's
stable repo...) *I would appreciate if someone from the KDE/plasma
team sponsor this package for inclusion to the stable repos at the
first practical opportunity; again, NOT for immediate inclusion into
KDE/plasma.* After months of field testing, perhaps it can
*/eventually/* be considered worthy enough for KDE extras...
I recommend reading [2] if you plan on maintaining this package yourself in
Debian.
Post by s***@osfda.org
Respectfully,
Steve Boriotti
Senior Developer, full stack
------------------------------------------------------------------------
About me: I am a developer with over 30 years commercial experience
(mostly in fintech...) Back in the 2000s I did code auditing of the
open source Cyrus mail server <https://www.cyrusimap.org> (using
"RATS
<https://code.google.com/archive/p/rough-auditing-tool-for-security>"...)
I also happen to be a postgres advocate, and have published a python
tool for generating self-signed certificates for pgadmin (a rather
hairy multistep process when done manually...)
[https://gitlab.com/osfda/pg_ssl_init] I even furnished patches on
StackOverflow to remedy bugs in the new asynchronous psycopg3
<https://www.psycopg.org/psycopg3> driver.
Best,
Antonio
[1]
https://docs.nextcloud.com/server/20/admin_manual/configuration_files/file_versioning.html
[2] https://www.debian.org/doc/manuals/maint-guide/
Andrey Rakhmatullin
2024-12-15 20:40:01 UTC
Permalink
Post by s***@osfda.org
I finally got the documentation for my package squared away, along with
support for _both_ gitlab and github interoperability (with github having a
much more idiosyncratic REST API...)
* My package's documentation is here: https://repliversion.org
* My package's repo is here: https://git.repliversion.org
* The .deb for my package is here: https://deb.repliversion.org
The deb for my package works, but since it is the first one I have ever
assembled, there is a good chance it has deficiencies.
What are the next steps to getting this into unstable?
If you want to do it yourself, follow
https://mentors.debian.net/intro-maintainers/ , learn how to make source
packages, make a correct Policy-compliant source package, ask for a
sponsor. Make sure you want and are able to maintain it after that.
If you want it to be done by someone else, there are no steps outside of
waiting until someone does it.
--
WBR, wRAR
s***@osfda.org
2024-12-15 21:20:01 UTC
Permalink
Will; do; I definitely am looking to maintain it. I can't imagine
someone taking over on ~10K lines of code.

Thanks...
Post by Andrey Rakhmatullin
Post by s***@osfda.org
I finally got the documentation for my package squared away, along with
support for _both_ gitlab and github interoperability (with github having a
much more idiosyncratic REST API...)
* My package's documentation is here: https://repliversion.org
* My package's repo is here: https://git.repliversion.org
* The .deb for my package is here: https://deb.repliversion.org
The deb for my package works, but since it is the first one I have ever
assembled, there is a good chance it has deficiencies.
What are the next steps to getting this into unstable?
If you want to do it yourself, follow
https://mentors.debian.net/intro-maintainers/ , learn how to make source
packages, make a correct Policy-compliant source package, ask for a
sponsor. Make sure you want and are able to maintain it after that.
If you want it to be done by someone else, there are no steps outside of
waiting until someone does it.
Martin Steigerwald
2024-10-04 08:50:01 UTC
Permalink
Hi Steve.

Thanks for your efforts.
Post by s***@osfda.org
I have a package that manages file versioning that has been alpha tested
* bookworm
* KDE plasma
Think like: Windows File versioning/Mac Revert. It is based on inotify.
[Before you think that couldn't be reliable, hear me out for a
discussion of safeguards I take to make it so -at some future point I
could eventually do the heavy lift to implement it as a shim to the EXT4
driver., but for now: it works..]
I wonder have you considered proposing your project to KDE upstream?

They could provide valuable review of your approach and also this way your
software would more easily be accessible to other Linux distributions as
well. Also they might have ideas on how to best integrate it with
components of their desktop.

I wonder especially as their Baloo desktop search already places an
inotify watches on every directory and sub directory that is included in
desktop search. (Also Akonadi AFAIK places inotify watches on local
maildir storage directories.)

This led me to set:

% cat /etc/sysctl.d/desktop.conf
fs.inotify.max_user_watches = 500000

As I have a huge lot of folders with files and a lot of folders in KMail.

Also regarding an Ext4 filesystem driver… I wonder whether such a
functionality could be placed at least partly within VFS layer so it can
be used in multiple filesystems. I do not use Ext4 except for /boot these
times. Almost all my filesystems are BTRFS currently and I am monitoring
BCacheFS and testing it for some scratch filesystem currently. Many also
may use XFS these times, especially with large file systems, also AFAIK it
is default with RHEL based distributions and maybe even still with
separate /home on SLES based distributions.

Just some thoughts and ideas on how to approach your project. Maybe you
thought about these already. Do as you wish with the ideas I provided. No
need to take time to reply to me about it.

All the best for your project.

Thanks again,
--
Martin
Loading...