Popular New Releases in Platform As A Service
herokuish
v0.5.34
jekyll-auth
heroku-buildpack-ruby
heroku/ruby 0.1.3
nova-backup-tool
5.0.1
spotlight
Popular Libraries in Platform As A Service
by AssetSync 
ruby
1819
Synchronises Assets between Rails and S3
by imwilsonxu python
1650
Fbone (Flask bone) is a Flask (Python microframework) starter/template/bootstrap/boilerplate application.
by piku python
1434 MIT
The tiniest PaaS you've ever seen. Piku allows you to do git push deployments to your own servers.
by gliderlabs shell
1301 MIT
Utility for emulating Heroku build and runtime tasks in containers
by ddollar ruby
1259
Multiple account management for Heroku
by heroku shell
1208 MIT
The official Heroku buildpack for Node.js apps.
by loklak shell
1119
heroku buildpack for Apache Ant to run Loklak
by heroku ruby
915 MIT
The official Heroku buildpack for Python apps.
by progrium groovy
913 MIT
Buildstep uses Docker and Buildpacks to build applications like Heroku
Trending New libraries in Platform As A Service
by leerob javascript
466
Easily add user reviews, feedback, and comments to your website in one line of code.
by yeahwu shell
418 Apache-2.0
by livewire-ui php
294 MIT
Livewire component that brings Spotlight/Alfred-like functionality to your Laravel application.
by tighten php
276
by mixool shell
208 Apache-2.0
websocket proxy on heroku
by moneymeets shell
157 MIT
Heroku buildpack that makes it possible to use Poetry with the official Python buildpack
by Mr-confused shell
120
by brighterman csharp
83
AbpVnext模块 设置管理+审计日志 +Saas+组织机构
by AcrobaticPanicc html
78 MIT
Stock portfolio manager web app built with Flask
Top Authors in Platform As A Service
1
129 Libraries
10345
2
18 Libraries
2454
3
14 Libraries
164
4
12 Libraries
95
5
11 Libraries
53
6
10 Libraries
223
7
9 Libraries
641
8
9 Libraries
166
9
8 Libraries
421
10
8 Libraries
64
1
129 Libraries
10345
2
18 Libraries
2454
3
14 Libraries
164
4
12 Libraries
95
5
11 Libraries
53
6
10 Libraries
223
7
9 Libraries
641
8
9 Libraries
166
9
8 Libraries
421
10
8 Libraries
64
Trending Kits in Platform As A Service
No Trending Kits are available at this moment for Platform As A Service
Trending Discussions on Platform As A Service
PostgreSQL: deadlock without a transaction
Using google cloud for image classification, cropping and OCR
QUESTION
PostgreSQL: deadlock without a transaction
Asked 2021-Apr-08 at 19:41I have a route (in a node JS app) that inserts and updates some data in a PostgreSQL database (version 13).
In pseudo-code, here are all queries that are done in sequential order:
1select * from x where y=$1;
2
3-- if there is no result
4 insert into x (y,z) values ($1,$2) returning *;
5
6-- else if there is a result
7 insert into x (y,z) values ($1,$2) returning *; -- values are different
8 update x set y=$1 where z=$2 returning *;
9 update x set a=$1 where b=$2 returning *;
10
11-- end if
12On some instances of the app without that much traffic that writes on their own table, I have many deadlocks. I don't understand why since there is no transaction, only simple inserts, and in some cases update. I use prepared statements (to avoid SQL injections) thanks to node postgres.
I never had deadlocks before (or maybe I did not notice them), so I don't understand why it can happened.
What could be the reason why there are deadlocks? And how to avoid them?
EDITLogs from the server (I don't have more detailed logs since it's on a platform as a service):
1select * from x where y=$1;
2
3-- if there is no result
4 insert into x (y,z) values ($1,$2) returning *;
5
6-- else if there is a result
7 insert into x (y,z) values ($1,$2) returning *; -- values are different
8 update x set y=$1 where z=$2 returning *;
9 update x set a=$1 where b=$2 returning *;
10
11-- end if
122021-04-07T11:55:17+02:00 Process 583773 waits for ShareLock on transaction 2408877; blocked by process 583789.
132021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] DETAIL: Process 583789 waits for ShareLock on transaction 2408880; blocked by process 583773.
142021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] ERROR: deadlock detected
152021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] STATEMENT: update x set user_id=$1, user_properties=$2 where user_id=$3
162021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] HINT: See server log for query details.
172021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] CONTEXT: while rechecking updated tuple (119,3) in relation "x"
182021-04-07T11:55:17+02:00 Process 583773: update x set user_id=$1, user_properties=$2 where user_id=$3
192021-04-07T11:55:17+02:00 Process 583789: update x set user_id=$1, user_properties=$2 where user_id=$3
20ANSWER
Answered 2021-Apr-08 at 19:41In PostgreSQL all data modification happens in a transaction. Even if it is only a single-statement transaction, there is still a transaction.
The log entry is not enough to give a definitive answer, but it sure looks like your updates are updating more than one row each. If they occasionally update the same rows but in different orders, they can dead lock against each other. I would think that that should probably be rare for the queries in your log, as I would think they would choose rows to update based on single-valued scan of the same index and so generally do it in the same order.
QUESTION
Using google cloud for image classification, cropping and OCR
Asked 2021-Feb-26 at 15:49Please allow me to ask a rather newbie question. So far, I have been using local tools like imagemagick or GOCR to perform the job, but that is rather old-fashioned, and I am urged to "move to google cloud AI".
The setupI have a (training) data set of various documents (as JPG and PDF) of different kinds, and by certain features (like prevailing color, repetitive layout) I intend to classify them, e.g. as invoice type 1, invoice type 2, not an invoice. In a 2nd step, I would like to OCR certain predefined areas of each document and extract e.g. the address of the company sending the invoice and the date.
- In a modern platform as a service (pass), I have already set up an UI where I can upload new files. These are then locally stored in a directory with filenames (or in a MongoDB). Meta info like upload timestamp, user, original file name is stored in a DB.
- The newly uploaded file should should then be submitted to google cloud which should do the classification step, and deliver back the label to be saved in the database.
- The document pages should be auto-cropped, i.e. black or white margins are removed, most probably with google cloud as well. The parameters of the crop should be persisted in the DB.
- In case it is e.g. an invoice, OCR should be performed (again by google cloud) for certain regions of the documents, e.g. a bounding box of spanning from the mid of the page to the right margin in the upper 10% of the cropped page. The results of the OCR should be again persisted locally.
I seem to be missing the correct search term to figure out how to do it with google cloud. Is there an google-API (e.g. REST), I can use to upload and which gives me back the results of steps 2 to 4?
ANSWER
Answered 2021-Feb-26 at 15:49I think that your best option here is to use Document AI (REST API and Libraries).
Using Document AI, you can:
- Convert images to text
- Classify documents
- Analyze and extract entities
Additionally, for your use case, we have a new Document AI feature that is still in preview and has limited access which is the Invoice parser.
Invoice parser is similar to Form parser but for invoices instead of forms. Check out the Invoice parser page and you will see what I mean by preview and limited access.
AFIK, there isn't any GCP tool for image edition.
Community Discussions contain sources that include Stack Exchange Network
Tutorials and Learning Resources in Platform As A Service
Tutorials and Learning Resources are not available at this moment for Platform As A Service
Share this Page
Get latest updates on Platform As A Service