json-machine | Efficient , easy-to-use , and fast PHP JSON stream parser | JSON Processing library
kandi X-RAY | json-machine Summary
kandi X-RAY | json-machine Summary
JSON Machine is an efficient, easy-to-use and fast JSON stream/pull/incremental/lazy (whatever you name it) parser based on generators developed for unpredictably long JSON streams or documents. Main features are:.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of json-machine
json-machine Key Features
json-machine Examples and Code Snippets
Community Discussions
Trending Discussions on json-machine
QUESTION
There is this library https://github.com/halaxa/json-machine
(PHP) that allows me to read huge JSON files without loading the entire file to memory. It works great and the way I read the JSON is like this:
ANSWER
Answered 2020-Sep-02 at 15:14This class already provides a generator, you should be able to do something like this:
QUESTION
The plan: I am coding for a web site on which the users need to upload large JSON files (50MB - 500MB+). These JSON files contain information about the users. The site must retrieve these information and after that store it on a MySQL database that I have created. I am using JSON machine (https://github.com/halaxa/json-machine) instead of JSON decode (https://www.php.net/manual/en/function.json-decode.php) to parse these large JSON files faster. So the process goes like this:
1) The user uploads his own large JSON file.
2) I parse his JSON and store his data temporarily on my machine.
3) I send the data from my machine to MySQL database.
(I have also code for the last two steps combined).
2-3) After uploading is finished I parse the large JSON and store the data directly to MySQL.
Now the problem: I want the users to be occupied only for the first step (the upload), after that I want to let them free and able to navigate to other pages of my website even if the other two (or one) step(s) haven't been completed yet. So my problem is the time. The last two steps might take some minutes or even a lot of hours and I don't want to keep users waiting after they finish the upload of their JSON file.
My solutions: When this problem came up I tried to find a way to run the last two processes on the background and I found this (php background process in windows environment) although it didn't work out. After that, I tried AJAX. I thought if the page does not reload, the user will be able to navigate, but that didn't work either. The page now does not reload when a user uploads a JSON but the user is still occupied for the whole duration of the three steps. Also, ignore_user_abort() (https://www.php.net/manual/en/function.ignore-user-abort.php) is not a solution because in my case I don't want to let the users abort the page, I want them to keep navigating.
I have to mention that the code is working properly, it gets users JSON files and after that it parses and stores everything to my MySQL database exactly like I want it to do, but it just keeps the users waiting for hours because of the thousands of inserts it has to execute for the database, especially if they have a very large JSON file. I have tried many things and I still don't know what I have to do to fix this problem. Αny kind of help is welcome!
...ANSWER
Answered 2020-Feb-06 at 10:40Well, after some more searching, I found out that this solution isn't the best, but it's close to what I used in my case. So I combined it with some examples of this and I changed it to that:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install json-machine
PHP requires the Visual C runtime (CRT). The Microsoft Visual C++ Redistributable for Visual Studio 2019 is suitable for all these PHP versions, see visualstudio.microsoft.com. You MUST download the x86 CRT for PHP x86 builds and the x64 CRT for PHP x64 builds. The CRT installer supports the /quiet and /norestart command-line switches, so you can also script it.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page