Paperless: Integration into owncloud

[torwag]

Why not integrating this into owncloud. I have a owncloud running for all other kind of data. It would make much more sense if I could scan into the owncloud database, making sure I have all the data together rather then having to use two separate web-frontends.
An (artificial) example:
One has a ODF-based form which I print out to let people fill in the form,
Scan the filled-in form and save it in the same folder like the ODF.
Find both original and filled in version

A (real) example:
Students hand over printed versions of their thesis.
I correct them on paper (still did not found a workflow, which allow me to do that as efficient as on paper)
I scan those had-made corrections and save it with the original files form before and after the correction in a single folder.

There are many possible scenarios where digital data blends over with printed/scanned data and which I would love to keep together.

[danielquinn]

Scanning something into a database is a very non-descript term, but I should say right up-front that I have no intention on coupling tightly with something as big and complex as OwnCloud. With that said though, some simple things can be done to do what I think you're looking for:

1. Consume from a directory on OwnCloud. This is already possible by editing /etc/paperless.conf:

PAPERLESS_CONSUMPTION_DIR="/path/to/OwnCloud/location/consume"

1. Store its encrypted documents to OwnCloud. This isn't difficult to do, as it would just require that I introduce a new variable into /etc/paperless.conf called something like PAPERLESS_MEDIA_DIR which would override the current reference to ../media/.
2. Store its database file on OwnCloud as well. Similarly to (2), this could be done just by setting up another variable so that we allow users to set the location of the db.sqlite3 file to somewhere on the user's OwnCloud share instead of the default ../data/.

So, if that's what you had in mind for "OwnCloud integration", then I can accept this one. If however you wanted something tighter, writing right to whatever OwnCloud uses for its own data store etc., then yeah, I don't see me writing that anytime soon.

[JaimeObregon]

While ownCloud has a plug-in interface which allows third parties to package functionality into an add-on, and even sort of a "market" for the distribution of these extensions, I don't think Paperless would fit as one of such, at least without a considerable effort that would, by the way, reduce the scope of Paperless to the scope of ownCloud users.

I know and heavily use ownCloud and your proposal is remarkable, but I don't think it would be easy to integrate the many dependecies of third party libraries Paperless requires into a packaged ownCloud extension without a considerable loss of focus and development effort. At least, for the time being.

[torwag]

I agree, maybe paperless could come up with a modular concept for which you
split frontend and backends.
The frontend is basically all and everything paperless is doing, whereas
the backends could be a own database and a own web-gui or simply using e.g.
webdav to copy files into an running owncloud instances.
That would enable both directions with relatively less effort.

On 8 April 2016 at 15:39, Jaime Gómez Obregón [email protected]
wrote:

While ownCloud has a plug-in interface which allows third parties to
package functionality into an add-on, and even sort of a "market" for the
distribution of these extensions, I don't think Paperless would fit as one
of such, at least without a considerable effort that would, by the way,
reduce the scope of Paperless to the scope of ownCloud users.

I know and heavily use ownCloud and your proposal is remarkable, but I
don't think it would be easy to integrate the many dependecies of third
party libraries Paperless requires into a packaged ownCloud extension
without a considerable loss of focus and development effort. At least, for
the time being.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/danielquinn/paperless/issues/106#issuecomment-207435178

[danielquinn]

Paperless is actually both a front & back system, and the backend is the much more complicated component. The backend includes the consumption engine, which reads from a directory and/or a mailbox, then writes to disk and to the database, using Django as the framework. Reworking all of that to write to an OwnCloud database is _way_ more work than I think you are understanding.

Additionally, as @JaimeObregon points out, coupling these two systems so closely considerably limits the portability of Paperless on systems not running OwnCloud. What's the value in reworking Paperless to write to an "own database" and then plug into an "own web-gui" when the primary use case for it to exist on systems not running OwnCloud?

I'm not saying that this can't be done. By all means, if you can issue a pull request that allows people to optionally plug their system into OwnCloud, I'm all for it, but from what I'm reading here, you're calling for a tighter coupling than I'm comfortable with.

[torwag]

@danielquinn Thanks for your answer, I somehow overlooked your first post. Which already says, after I looked a bit into the source code and documentation, what I was going to kindly suggest ;) I am totally happy with paperless being standalone. Maybe many owncloud users might be happy with a "bridge", which in the easiest case is just a little how-to about "getting owncloud and paperless working side by side together".
Owncloud offers to add "external pages" to its menu. Thus, if the pdfs can be found in owncloud and one can easily call the paperless webfrontend from within owncloud the integration is as good as it need to be. Again my main reason for this feature request was not to have one single monolithic application but to avoid that I have to deal with two sets of data (pdfs) creating in the worst case even unneeded redundancy .

One question remains, with PDF encryption. If I understand this correctly, in the current form, owncloud could not open or read the pdfs due to encryption?. Would it be possible to deactivate encryption, e.g. by specifying "None" as passphrase?

Although, as I understand your solution, one should not move or copy PDF files from the paperless folder right? How about subfolders?

Thanks for the software and the support :)

[danielquinn]

Yeah it's sort of a rough fit, because in order to be useful at all in conjunction with OwnCloud, you'd have to disable encryption, change the location of document storage, and probably find a better way to access the index than the existing web interface. It might be a better model to use a different software suite to index and rewrite PDF files to include the OCR'd text, which you could then index in some other way inside OwnCloud.

Paperless' primary model is: "read the text of a document into the database to make it searchable" and everything after that is essentially gravy. I'm just convinced this model is compatible with OwnCloud in general. Perhaps something like OCRmyPDF is a better choice for what you're looking for?

[torwag]

Actually, I came across OCRmyPDF https://github.com/jbarlow83/OCRmyPDF by
reading about paperless. I gave it a try, it runs in a docker container.
Thus, it might require relatively less code to get a similar workflow as
paperless provides directly in owncloud. Scan into owncloud, run OCRmyPDF,
use owncloud lucene search to find text in pdfs

I will look into this

Thanks

On 12 April 2016 at 15:45, Daniel Quinn [email protected] wrote:

Yeah it's sort of a rough fit, because in order to be useful at all in
conjunction with OwnCloud, you'd have to disable encryption, change the
location of document storage, and probably find a better way to access the
index than the existing web interface. It might be a better model to use a
different software suite to index and rewrite PDF files to include the
OCR'd text, which you could then index in some other way inside OwnCloud.

Paperless' primary model is: "read the text of a document into the
database to make it searchable" and everything after that is essentially
gravy. I'm just convinced this model is compatible with OwnCloud in
general. Perhaps something like OCRmyPDF
https://github.com/jbarlow83/OCRmyPDF is a better choice for what
you're looking for?

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/danielquinn/paperless/issues/106#issuecomment-208913334

[Gummibeer]

I think that some more configuration options will be worth the work and solve a lot of issues. Coupling it to ownCloud is too much work and more a job for a second packe that uses the REST-API and binds these things to ownCloud. Otherwise there are a lot of extendable Storagesystems and everyone want's his one to be integrated and supported.
But with linux, symlinks and customizable paths it's simple to store the files in ownCloud, Synology or wherever you want.

[stgarf]

@torwag Anything change over the last three years or so? I'm just going through old issues in the repo to see what can potentially be worked on still or cleaned up. Please close out if no longer relevant.