Explore all Platform As A Service open source software, libraries, packages, source code, cloud functions and APIs.

Popular New Releases in Platform As A Service

herokuish

v0.5.34

jekyll-auth

heroku-buildpack-ruby

heroku/ruby 0.1.3

nova-backup-tool

5.0.1

spotlight

Popular Libraries in Platform As A Service

asset_sync

by AssetSync doticonrubydoticon

star image 1819 doticon

Synchronises Assets between Rails and S3

fbone

by imwilsonxu doticonpythondoticon

star image 1650 doticon

Fbone (Flask bone) is a Flask (Python microframework) starter/template/bootstrap/boilerplate application.

piku

by piku doticonpythondoticon

star image 1434 doticonMIT

The tiniest PaaS you've ever seen. Piku allows you to do git push deployments to your own servers.

herokuish

by gliderlabs doticonshelldoticon

star image 1301 doticonMIT

Utility for emulating Heroku build and runtime tasks in containers

heroku-accounts

by ddollar doticonrubydoticon

star image 1259 doticon

Multiple account management for Heroku

heroku-buildpack-nodejs

by heroku doticonshelldoticon

star image 1208 doticonMIT

The official Heroku buildpack for Node.js apps.

loklak_heroku_ant_buildpack

by loklak doticonshelldoticon

star image 1119 doticon

heroku buildpack for Apache Ant to run Loklak

heroku-buildpack-python

by heroku doticonrubydoticon

star image 915 doticonMIT

The official Heroku buildpack for Python apps.

buildstep

by progrium doticongroovydoticon

star image 913 doticonMIT

Buildstep uses Docker and Buildpacks to build applications like Heroku

Trending New libraries in Platform As A Service

fastfeedback

by leerob doticonjavascriptdoticon

star image 466 doticon

Easily add user reviews, feedback, and comments to your website in one line of code.

kinto

by yeahwu doticonshelldoticon

star image 418 doticonApache-2.0

spotlight

by livewire-ui doticonphpdoticon

star image 294 doticonMIT

Livewire component that brings Spotlight/Alfred-like functionality to your Laravel application.

novapackages

by tighten doticonphpdoticon

star image 276 doticon

kuhero

by mixool doticonshelldoticon

star image 208 doticonApache-2.0

websocket proxy on heroku

python-poetry-buildpack

by moneymeets doticonshelldoticon

star image 157 doticonMIT

Heroku buildpack that makes it possible to use Poetry with the official Python buildpack

nekopack

by Mr-confused doticonshelldoticon

star image 120 doticon

AbpVnextModule

by brighterman doticoncsharpdoticon

star image 83 doticon

AbpVnext模块 设置管理+审计日志 +Saas+组织机构

stocks-manager

by AcrobaticPanicc doticonhtmldoticon

star image 78 doticonMIT

Stock portfolio manager web app built with Flask

Top Authors in Platform As A Service

1

heroku

129 Libraries

star icon10345

2

ddollar

18 Libraries

star icon2454

3

hone

14 Libraries

star icon164

4

jamesward

12 Libraries

star icon95

5

lstoll

11 Libraries

star icon53

6

jkutner

10 Libraries

star icon223

7

optimistdigital

9 Libraries

star icon641

8

heroku-examples

9 Libraries

star icon166

9

dcasia

8 Libraries

star icon421

10

adamwiggins

8 Libraries

star icon64

1

129 Libraries

star icon10345

2

18 Libraries

star icon2454

3

14 Libraries

star icon164

4

12 Libraries

star icon95

5

11 Libraries

star icon53

6

10 Libraries

star icon223

7

9 Libraries

star icon641

8

9 Libraries

star icon166

9

8 Libraries

star icon421

10

8 Libraries

star icon64

Trending Kits in Platform As A Service

No Trending Kits are available at this moment for Platform As A Service

Trending Discussions on Platform As A Service

PostgreSQL: deadlock without a transaction

Using google cloud for image classification, cropping and OCR

QUESTION

PostgreSQL: deadlock without a transaction

Asked 2021-Apr-08 at 19:41

I have a route (in a node JS app) that inserts and updates some data in a PostgreSQL database (version 13).

In pseudo-code, here are all queries that are done in sequential order:

1select * from x where y=$1;
2
3-- if there is no result
4   insert into x (y,z) values ($1,$2) returning *;
5
6-- else if there is a result
7   insert into x (y,z) values ($1,$2) returning *; -- values are different
8   update x set y=$1 where z=$2 returning *;
9   update x set a=$1 where b=$2 returning *;
10
11-- end if
12

On some instances of the app without that much traffic that writes on their own table, I have many deadlocks. I don't understand why since there is no transaction, only simple inserts, and in some cases update. I use prepared statements (to avoid SQL injections) thanks to node postgres.

I never had deadlocks before (or maybe I did not notice them), so I don't understand why it can happened.

What could be the reason why there are deadlocks? And how to avoid them?

EDIT

Logs from the server (I don't have more detailed logs since it's on a platform as a service):

1select * from x where y=$1;
2
3-- if there is no result
4   insert into x (y,z) values ($1,$2) returning *;
5
6-- else if there is a result
7   insert into x (y,z) values ($1,$2) returning *; -- values are different
8   update x set y=$1 where z=$2 returning *;
9   update x set a=$1 where b=$2 returning *;
10
11-- end if
122021-04-07T11:55:17+02:00 Process 583773 waits for ShareLock on transaction 2408877; blocked by process 583789.
132021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] DETAIL: Process 583789 waits for ShareLock on transaction 2408880; blocked by process 583773.
142021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] ERROR: deadlock detected
152021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] STATEMENT: update x set user_id=$1, user_properties=$2 where user_id=$3
162021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] HINT: See server log for query details.
172021-04-07T11:55:17+02:00 2021-04-07 09:55:17.084 GMT [583789] CONTEXT: while rechecking updated tuple (119,3) in relation "x"
182021-04-07T11:55:17+02:00 Process 583773: update x set user_id=$1, user_properties=$2 where user_id=$3
192021-04-07T11:55:17+02:00 Process 583789: update x set user_id=$1, user_properties=$2 where user_id=$3
20

ANSWER

Answered 2021-Apr-08 at 19:41

In PostgreSQL all data modification happens in a transaction. Even if it is only a single-statement transaction, there is still a transaction.

The log entry is not enough to give a definitive answer, but it sure looks like your updates are updating more than one row each. If they occasionally update the same rows but in different orders, they can dead lock against each other. I would think that that should probably be rare for the queries in your log, as I would think they would choose rows to update based on single-valued scan of the same index and so generally do it in the same order.

Source https://stackoverflow.com/questions/66968214

QUESTION

Using google cloud for image classification, cropping and OCR

Asked 2021-Feb-26 at 15:49

Please allow me to ask a rather newbie question. So far, I have been using local tools like imagemagick or GOCR to perform the job, but that is rather old-fashioned, and I am urged to "move to google cloud AI".

The setup

I have a (training) data set of various documents (as JPG and PDF) of different kinds, and by certain features (like prevailing color, repetitive layout) I intend to classify them, e.g. as invoice type 1, invoice type 2, not an invoice. In a 2nd step, I would like to OCR certain predefined areas of each document and extract e.g. the address of the company sending the invoice and the date.

The architecture I am envisioning
  1. In a modern platform as a service (), I have already set up an UI where I can upload new files. These are then locally stored in a directory with filenames (or in a MongoDB). Meta info like upload timestamp, user, original file name is stored in a DB.
  2. The newly uploaded file should should then be submitted to google cloud which should do the classification step, and deliver back the label to be saved in the database.
  3. The document pages should be auto-cropped, i.e. black or white margins are removed, most probably with google cloud as well. The parameters of the crop should be persisted in the DB.
  4. In case it is e.g. an invoice, OCR should be performed (again by google cloud) for certain regions of the documents, e.g. a bounding box of spanning from the mid of the page to the right margin in the upper 10% of the cropped page. The results of the OCR should be again persisted locally.
The problem

I seem to be missing the correct search term to figure out how to do it with google cloud. Is there an google-API (e.g. REST), I can use to upload and which gives me back the results of steps 2 to 4?

ANSWER

Answered 2021-Feb-26 at 15:49

I think that your best option here is to use Document AI (REST API and Libraries).

Using Document AI, you can:

  • Convert images to text
  • Classify documents
  • Analyze and extract entities

Additionally, for your use case, we have a new Document AI feature that is still in preview and has limited access which is the Invoice parser.

Invoice parser is similar to Form parser but for invoices instead of forms. Check out the Invoice parser page and you will see what I mean by preview and limited access.

AFIK, there isn't any GCP tool for image edition.

Source https://stackoverflow.com/questions/66348794

Community Discussions contain sources that include Stack Exchange Network

Tutorials and Learning Resources in Platform As A Service

Tutorials and Learning Resources are not available at this moment for Platform As A Service

Share this Page

share link

Get latest updates on Platform As A Service