pdftotext | Simple PDF text extraction | Document Editor library

I would recommend to downgrade your milvus version to a version before the 2.0 release just a week ago. Here is a discussion on that topic: https://github.com/deepset-ai/haystack/issues/2081

Short answer, I bet you have a space in your Window's account name (say Your Account is where your account is saved so you have C:\Users\Your Account folder, and there is also a text file C:\Users\Your ("Your" being the first part of your user name). MSVS2022 (maybe earlier versions, too) is known to leave this log file which exposes a bug in Python venv's python launcher. Delete this text file, and your problem should be solved.

See my question/answer for more details.

The code convention whilst printing Plain Text is that FF usually means FormFeed it is a Control Code to the printer

↑ 12 00/12 14 %0C FF (CtrL=^L) FORM FEED (Page Break)

This is a way to indicate / eject an End Of Page, so you should see one at the division between pages.

There is a switch to remove/exclude them so try ,

Just match a continuous series of single letters separated by spaces, then delete all spaces from that using a nested substitution (the /e eval modifier).

This program was meant to be ran in linux, so there are a few steps you need to do in order for it to run in windows.

1. Install the libraries.

Installation in Anaconda:

To capture all output across all convert-PDFtoText in a single output file, use a single pipeline with the ForEach-Object cmdlet:

I took the advice in the comments. I did not use pandas, used list comprehension, and rewrote this as:

To answer the direct question what is different is the CID data so lets just look at one object on each page 1. here I pick the subject of your question, the first text that includes the numbers 1 2 9 0, letters L E G I S A T U R and the others in title

Here we see good or bad they are all stored as the same font type ??????+PSOwstnewcspsb, unclear to me but seems to be named along the lines PSO WeSTern NEW Courier ??? Bold

So why would there then be some working as mapped correctly by say OCR and some not ? That is an unknown to me and there is often no clear rhyme or reason, but we can see a difference in outcomes as the good one starts with printable space (/FirstChar 32/LastChar 116) whilst both of the non working ones start (/FirstChar 0/LastChar ## of approx 66) i.e. include a non standard printing range. That however is not an indicator of a bad font and in other bad examples I have seen /FirstChar 2 as giving a hint to a poorly defined font. the problem with searching /FirstChar is it may be encrypted or encode thus not possible to look for in many pdfs until disassembled.

The only good indication of bad characters is good plain text extraction contains invalid print characters.

You say you wish to avoid files with bad construct but many files may only have bad parts of pages, for a wider example of this issue see How to identify likely broken pdf pages before extracting its text?

The pdf, here . I tried to open with PyDF4 you got an annoying problem, PdfReadWarning: Superfluous whitespace found in object header [...] which is still unsolved with PyPDF, probably due to bad quality of the file. So I converted to text from the shell using pdftotext.

I tried to find the start index of each block, with a regex criteria. Automatically you have also the end, which is the index of the next block (minus 1).

Once you have start and end index the corresponding slice will be a block.