Tesseract: Significant speed drop on Tesseract4 vs 3 with identical image

[mattmillus]

Environment

• Tesseract Version: 3.51 and 4.00
• Commit Number: 10a8a67 (when built from source), also windows binaries from https://github.com/tesseract-ocr/tesseract/wiki/Downloads
• Platform: Windows7 64-bit, 32GB of RAM, Intel Core i7-4770K @ 3.50Ghz

Current Behavior:

Using the same input image (attached), Tesseract4 performs over three times slower than Tesseract3. On my system this image took ~5 seconds with Tesseract3, and ~20 seconds with Tesseract4.

Tested using the pre-built 4.00 and 3.51 windows binaries from https://github.com/tesseract-ocr/tesseract/wiki/Downloads, as well as with the latest 4.00 git source I built locally using cppan with MSVC 2015, in Release config

Using the 'fast' version of eng.traineddata did not make a significant difference.

Expected Behavior:

https://github.com/tesseract-ocr/tesseract/wiki/4.0-Accuracy-and-Performance implies we should not be expecting such significantly slower results

EDIT: The following image is one of a handful of test images I am using. Zlib isn't relevant here.

[amitdo]

How zlib is related here?

[mattmillus]

It is not related at all, that is simply a well-formed sample image I was using to test the performance. Performance is similar with other images of similar dimensions

[amitdo]

Unlike the Linux build, OpenMP and AVX2 are currently not activated in the Windows build.

[mattmillus]

Thank you for the info. I added /openmp in Visual Studio as per https://msdn.microsoft.com/en-us/library/fw509c3b.aspx - this nearly doubled the speed and now puts the process at 50% CPU usage. On windows Tesseract3 is still almost twice as fast, but at least this helped!

[Shreeshrii]

Also see https://github.com/tesseract-ocr/tesseract/issues/898

[amitdo]

CC: @egorpugin

[jbreiden]

I see wall time of about 4.5 seconds on a six core Xeon E5-1650 running at 3.2 Ghz under Linux. About 3.5 seconds under optimal throughput conditions (repeated OCR of the same image over and over).

[Shreeshrii]

Also see https://github.com/tesseract-ocr/tesseract/wiki/NeuralNetsInTesseract4.00#hardware-and-cpu-requirements

On a machine with multiple cores, and AVX, an easy English image may take twice as much real time, and use 7 times the CPU as base Tesseract, whereas Hindi takes more CPU than base Tesseract, but actually runs faster in terms of real time.

[amitdo]

https://github.com/tesseract-ocr/tesseract/issues/943#issuecomment-303239929

theraysmith commented on May 23, 2017

*Far greater performance improvements can be made by making the network smaller.* As I already indicated, I have had some very good results in this area, with a network 3x faster than the legacy code (for English) and much faster than the legacy code for complex scripts.

https://github.com/tesseract-ocr/tesseract/issues/995#issuecomment-314609036

theraysmith commented on Jul 12, 2017

2 parallel sets of tessdata. "best" and "fast". "Fast" will exceed the speed of legacy Tesseract in real time, provided you have the required parallelism components, and in total CPU only slightly slower for English. Way faster for most non-latin languages, while being <5% worse than "best".

[mattmillus]

I did further testing and the plot thickens a bit.. I ran Tesseract4 with the official datafiles that support the legacy engine (from https://github.com/tesseract-ocr/tessdata) and set EngineMode to TESSERACT so that only the legacy engine is running, not LSTM. I then compared the wall time on a 200 page image file to that of Tesseract 3.02, and found that Tesseract4.00 took over 50% longer (260 seconds vs 160 seconds), despite the fact that its should be using the same engine??

[amitdo]

That's another issue.

https://github.com/tesseract-ocr/tesseract/issues/263#issuecomment-198463943
https://groups.google.com/forum/?hl=en#!topic/tesseract-dev/LErriuT-sck

[mattmillus]

In ref to the quote above:

2 parallel sets of tessdata. "best" and "fast". "Fast" will exceed the speed of legacy Tesseract in real time, provided you have the required parallelism components, and in total CPU only slightly slower for English.

I have not found the above to be true, at least on windows with English, here are some results:
711 seconds with Tesseract4.00, "fast" data, no /openMP
353 seconds with Tesseract4.00, "fast" data, with /openMP on
480 seconds with Tesseract4.00, "best" data, with /openMP on
260 seconds with Tesseract4.00 "legacy" data and legacy engine, with /openMP on (though it does nothing for legacy engine)
160 seconds with Tesserac3.02

Do I need to manually turn on AVX for windows builds somehow?

[mattmillus]

Interesting.. so you think that if I test with Tesseract3.05 it will be about the same speed as 4.00 with legacy engine, and significantly slower than 3.02? If so thats very unfortunate... why such a change if speed with 3.02 vs 3.05 if they the same engine?

[amitdo]

https://github.com/tesseract-ocr/tesseract/blob/a538cd126ba126d05c9e6cb3b11361f0808e19b3/CMakeLists.txt#L210

AVX is turn on but AVX2 is not.

[Shreeshrii]

Ray/Jeff need to confirm that the files in tessdata_fast are indeed the final fast models that Ray was referring to.

[mattmillus]

If not I would love to test with the final fast models!

[amitdo]

The CMake file is incomplete.

[jbreiden]

The fast models on Github are final. (To the degree that anything in life is final.)

[amitdo]

Pinging @egorpugin again.

[egorpugin]

So, do you want /openmp for the whole tess or only for specific files?

[amitdo]

With autotools it is global.

[egorpugin]

Done. https://github.com/tesseract-ocr/tesseract/commit/4b6fefb2ac8cd2cd82ff492ee91710f127580211

[amitdo]

Thanks Egor.

Please also add avx2 flag to arch/intsimdmatrixavx2.cpp,
and sse4.1 flag to arch/intsimdmatrixsse.cpp

This is needed to make the use of traineddata from the 'fast' repo actually faster than 'best'.

[egorpugin]

Done. https://github.com/tesseract-ocr/tesseract/commit/2da95d63bc7db973f0dbb250f391da3652339772
Also note these's an ICE with MSVC in arch/intsimdmatrixavx2.cpp.
I've added simple workaround as in arch/dotproductavx.cpp.
Now I'd like someone to test my changes to check if speedup is actually turned on.

MS issue for reference. Feel free to upvote.
https://developercommunity.visualstudio.com/content/problem/187655/clexe-vs155-ice-with-avx2-flag-on-mm256-extract-ep.html

[stweil]

Does anybody know how a GCC based build (for example from UB-Mannheim) compares with the MSVC results?

[amitdo]

1290

[amitdo]

Here are the current options to run Tesseract on Windows:

• 'Native' (Win32 API)
  
  
  
  • MSVC
  
  
  • MinGW-w64
  
  
• Cygwin (Posix API)
• MS WSL (Win 10)
• VM like VirtualBox
• Docker

And they say that on Linux you have too many options to choose from, which confuses people...
:-)

[stweil]

There is also the choice of building 32 or 64 bit software for Windows. Personally I typically built 32 bit Tesseract for Windows up to now, but maybe there might be good reasons for 64 bit code?

[innir]

Just to add some information from a Debian bug report, without AVX2 tesseract 4 is much slower than 3.05 ... (See also https://github.com/manisandro/gImageReader/issues/285 for another example of the slowdown)

[amitdo]

If someone prefers speed over accuracy, Tesseract 4.00 still includes the legacy ocr engine; Use --oem 0 with the tessdata traineddata.

[Shreeshrii]

@zdenop Please label

Performance

[Shreeshrii]

Post in forum by David Tran

Server performance is 3x as slow versus local machine 1 post by 1 author

Local machine: 3.50Ghz, 16 GB Ram, Windows 7 64 bit
Server: 2.30Ghz, 32 GB Ram, WindowServer2012 64 bit

tesseract v4.0.0-beta.4.20180912 64 bit

Current Behavior: Processing a 64page PDF (2,733KB) on my local machine takes 286 seconds while on our server it takes a whopping 842 seconds.

Expected Behavior: That it wouldn't be 3x as slow on the server versus on my local machine.

What could be the root cause of the degradation in performance?

[stweil]

It looks like the local machine is rather new hardware, while the server is older. So it could be AVX / SSE none at all. The user can run tesseract --version on both machines to see whether SSE and AVX are found.

The number of CPU cores and the memory bandwidth are also very important.
And of course it makes a difference if there are other processes running in parallel on the server.

The user uses the UB Mannheim installer for Windows. He should update to the latest version.

[ripefig]

AVX2 and SSE are found but performance is terrible on my machine. It takes over a minute to OCR a single sentence. Tesseract 3 did it about a second, tesseract 4 is not usable on my machine.
https://github.com/danpla/dpscreenocr/issues/2

[zdenop]

Closing as duplicate to #263.