Interpreter

Explore all libraries in Interpreter

Explore all Interpreter open source software, libraries, packages, source code, cloud functions and APIs.

Popular New Releases in Interpreter

micropython

F-strings, new machine.I2S class, ESP32-C3 support and LEGO_HUB_NO6 board

yaegi

v0.11.3

vm2

3.9.9

boa

v0.14

Popular Libraries in Interpreter

by v8 c++

17859 NOASSERTION

The official mirror of the V8 Git repository

micropython

by micropython c

13567 NOASSERTION

MicroPython - a lean and efficient Python implementation for microcontrollers and constrained systems

RustPython

by RustPython rust

9450 NOASSERTION

A Python Interpreter written in Rust

otto

by robertkrimen go

6194 MIT

A JavaScript interpreter in Go (golang)

by mvdan go

4647 BSD-3-Clause

A shell parser, formatter, and interpreter with bash support; includes shfmt

groovy

by apache java

4480 Apache-2.0

Apache Groovy: A powerful multi-faceted programming language for the JVM platform

yaegi

by traefik go

4279 Apache-2.0

Yaegi is Another Elegant Go Interpreter

craftinginterpreters

by munificent html

3851 NOASSERTION

Repository for the book "Crafting Interpreters"

wuffs

by google c

2988 Apache-2.0

Wrangling Untrusted File Formats Safely

Explore all libraries in Interpreter

Trending New libraries in Interpreter

pua-lang

by flaneur2020 rust

1862 MIT

hello-world.rs

by mTvare6 rust

1695 NOASSERTION

🚀Memory safe, blazing fast, configurable, minimal hello world written in rust(🚀) in a few lines of code with few(1092🚀) dependencies🚀

pocketlang

by ThakeeNathees c

1315 MIT

A lightweight, fast embeddable scripting language.

Cantonese

by StepfenShawn python

977 MIT

粤语编程语言.The Cantonese programming language.

lisp

by robpike go

818 BSD-3-Clause

Toy Lisp 1.5 interpreter

sablejs

by sablejs javascript

775 NOASSERTION

🏖️ The safer and faster ECMA5.1 interpreter written by JavaScript

rhizome

by chrisseaton ruby

669 MIT

A JIT for Ruby, implemented in pure Ruby

passerine

by vrtbl rust

384 MIT

A small extensible programming language designed for concise expression with little code.

makeJs

by zuluoaaa javascript

305 MIT

A sub Javascript interpreter for interpreting itself

Top Authors in Interpreter

ajlopez

22 Libraries

133

cslarsen

6 Libraries

540

howerj

5 Libraries

137

hashemi

5 Libraries

nukata

5 Libraries

m-ender

5 Libraries

350

catseye

5 Libraries

codr7

5 Libraries

406

thautwarm

4 Libraries

133

PacktPublishing

4 Libraries

154

ajlopez

22 Libraries

133

cslarsen

6 Libraries

540

howerj

5 Libraries

137

hashemi

5 Libraries

nukata

5 Libraries

m-ender

5 Libraries

350

catseye

5 Libraries

codr7

5 Libraries

406

thautwarm

4 Libraries

133

PacktPublishing

4 Libraries

154

Trending Kits in Interpreter

No Trending Kits are available at this moment for Interpreter

Trending Discussions on Interpreter

spark-shell throws java.lang.reflect.InvocationTargetException on running

ImportError: cannot import name 'safe_str_cmp' from 'werkzeug.security'

What issue could I have in Gradle managed device setup?

How to compose Free Monads

Running one python script within another script using subprocess

How is CPython implemented?

Stricter than strict mode?

How to stop PyCharm's break/stop/halt feature on handled exceptions (i.e. only break on python unhandled exceptions)?

Wrong PHP Version/Executable in VSCode terminal but works perfectly in Mac terminal

Why does this Python code with threading have race conditions?

spark-shell throws java.lang.reflect.InvocationTargetException on running

ImportError: cannot import name 'safe_str_cmp' from 'werkzeug.security'

What issue could I have in Gradle managed device setup?

How to compose Free Monads

Running one python script within another script using subprocess

How is CPython implemented?

Stricter than strict mode?

How to stop PyCharm's break/stop/halt feature on handled exceptions (i.e. only break on python unhandled exceptions)?

Wrong PHP Version/Executable in VSCode terminal but works perfectly in Mac terminal

Why does this Python code with threading have race conditions?

QUESTION

spark-shell throws java.lang.reflect.InvocationTargetException on running

Asked 2022-Apr-01 at 19:53

When I execute run-example SparkPi, for example, it works perfectly, but when I run spark-shell, it throws these exceptions:

1WARNING: An illegal reflective access operation has occurred
2WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/C:/big_data/spark-3.2.0-bin-hadoop3.2-scala2.13/jars/spark-unsafe_2.13-3.2.0.jar) to constructor java.nio.DirectByteBuffer(long,int)
3WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
4WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
5WARNING: All illegal access operations will be denied in a future release
6Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
7Setting default log level to &quot;WARN&quot;.
8To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
9Welcome to
10      ____              __
11     / __/__  ___ _____/ /__
12    _\ \/ _ \/ _ `/ __/  '_/
13   /___/ .__/\_,_/_/ /_/\_\   version 3.2.0
14      /_/
15
16Using Scala version 2.13.5 (OpenJDK 64-Bit Server VM, Java 11.0.9.1)
17Type in expressions to have them evaluated.
18Type :help for more information.
1921/12/11 19:28:36 ERROR SparkContext: Error initializing SparkContext.
20java.lang.reflect.InvocationTargetException
21        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
22        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
23        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
24        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
25        at org.apache.spark.executor.Executor.addReplClassLoaderIfNeeded(Executor.scala:909)
26        at org.apache.spark.executor.Executor.&lt;init&gt;(Executor.scala:160)
27        at org.apache.spark.scheduler.local.LocalEndpoint.&lt;init&gt;(LocalSchedulerBackend.scala:64)
28        at org.apache.spark.scheduler.local.LocalSchedulerBackend.start(LocalSchedulerBackend.scala:132)
29        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:220)
30        at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:581)
31        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)
32        at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)
33        at scala.Option.getOrElse(Option.scala:201)
34        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)
35        at org.apache.spark.repl.Main$.createSparkSession(Main.scala:114)
36        at $line3.$read$$iw.&lt;init&gt;(&lt;console&gt;:5)
37        at $line3.$read.&lt;init&gt;(&lt;console&gt;:4)
38        at $line3.$read$.&lt;clinit&gt;(&lt;console&gt;)
39        at $line3.$eval$.$print$lzycompute(&lt;synthetic&gt;:6)
40        at $line3.$eval$.$print(&lt;synthetic&gt;:5)
41        at $line3.$eval.$print(&lt;synthetic&gt;)
42        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
43        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
44        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
45        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
46        at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:670)
47        at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1006)
48        at scala.tools.nsc.interpreter.IMain.$anonfun$doInterpret$1(IMain.scala:506)
49        at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)
50        at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)
51        at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:43)
52        at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:505)
53        at scala.tools.nsc.interpreter.IMain.$anonfun$doInterpret$3(IMain.scala:519)
54        at scala.tools.nsc.interpreter.IMain.doInterpret(IMain.scala:519)
55        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:503)
56        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:501)
57        at scala.tools.nsc.interpreter.IMain.$anonfun$quietRun$1(IMain.scala:216)
58        at scala.tools.nsc.interpreter.shell.ReplReporterImpl.withoutPrintingResults(Reporter.scala:64)
59        at scala.tools.nsc.interpreter.IMain.quietRun(IMain.scala:216)
60        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$interpretPreamble$1(ILoop.scala:924)
61        at scala.collection.immutable.List.foreach(List.scala:333)
62        at scala.tools.nsc.interpreter.shell.ILoop.interpretPreamble(ILoop.scala:924)
63        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$run$3(ILoop.scala:963)
64        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
65        at scala.tools.nsc.interpreter.shell.ILoop.echoOff(ILoop.scala:90)
66        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$run$2(ILoop.scala:963)
67        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
68        at scala.tools.nsc.interpreter.IMain.withSuppressedSettings(IMain.scala:1406)
69        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$run$1(ILoop.scala:954)
70        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
71        at scala.tools.nsc.interpreter.shell.ReplReporterImpl.withoutPrintingResults(Reporter.scala:64)
72        at scala.tools.nsc.interpreter.shell.ILoop.run(ILoop.scala:954)
73        at org.apache.spark.repl.Main$.doMain(Main.scala:84)
74        at org.apache.spark.repl.Main$.main(Main.scala:59)
75        at org.apache.spark.repl.Main.main(Main.scala)
76        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
77        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
78        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
79        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
80        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
81        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
82        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
83        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
84        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
85        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
86        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
87        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
88Caused by: java.net.URISyntaxException: Illegal character in path at index 42: spark://DESKTOP-JO73CF4.mshome.net:2103/C:\classes
89        at java.base/java.net.URI$Parser.fail(URI.java:2913)
90        at java.base/java.net.URI$Parser.checkChars(URI.java:3084)
91        at java.base/java.net.URI$Parser.parseHierarchical(URI.java:3166)
92        at java.base/java.net.URI$Parser.parse(URI.java:3114)
93        at java.base/java.net.URI.&lt;init&gt;(URI.java:600)
94        at org.apache.spark.repl.ExecutorClassLoader.&lt;init&gt;(ExecutorClassLoader.scala:57)
95        ... 67 more
9621/12/11 19:28:36 ERROR Utils: Uncaught exception in thread main
97java.lang.NullPointerException
98        at org.apache.spark.scheduler.local.LocalSchedulerBackend.org$apache$spark$scheduler$local$LocalSchedulerBackend$$stop(LocalSchedulerBackend.scala:173)
99        at org.apache.spark.scheduler.local.LocalSchedulerBackend.stop(LocalSchedulerBackend.scala:144)
100        at org.apache.spark.scheduler.TaskSchedulerImpl.stop(TaskSchedulerImpl.scala:927)
101        at org.apache.spark.scheduler.DAGScheduler.stop(DAGScheduler.scala:2516)
102        at org.apache.spark.SparkContext.$anonfun$stop$12(SparkContext.scala:2086)
103        at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1442)
104        at org.apache.spark.SparkContext.stop(SparkContext.scala:2086)
105        at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:677)
106        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)
107        at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)
108        at scala.Option.getOrElse(Option.scala:201)
109        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)
110        at org.apache.spark.repl.Main$.createSparkSession(Main.scala:114)
111        at $line3.$read$$iw.&lt;init&gt;(&lt;console&gt;:5)
112        at $line3.$read.&lt;init&gt;(&lt;console&gt;:4)
113        at $line3.$read$.&lt;clinit&gt;(&lt;console&gt;)
114        at $line3.$eval$.$print$lzycompute(&lt;synthetic&gt;:6)
115        at $line3.$eval$.$print(&lt;synthetic&gt;:5)
116        at $line3.$eval.$print(&lt;synthetic&gt;)
117        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
118        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
119        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
120        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
121        at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:670)
122        at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1006)
123        at scala.tools.nsc.interpreter.IMain.$anonfun$doInterpret$1(IMain.scala:506)
124        at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)
125        at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)
126        at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:43)
127        at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:505)
128        at scala.tools.nsc.interpreter.IMain.$anonfun$doInterpret$3(IMain.scala:519)
129        at scala.tools.nsc.interpreter.IMain.doInterpret(IMain.scala:519)
130        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:503)
131        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:501)
132        at scala.tools.nsc.interpreter.IMain.$anonfun$quietRun$1(IMain.scala:216)
133        at scala.tools.nsc.interpreter.shell.ReplReporterImpl.withoutPrintingResults(Reporter.scala:64)
134        at scala.tools.nsc.interpreter.IMain.quietRun(IMain.scala:216)
135        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$interpretPreamble$1(ILoop.scala:924)
136        at scala.collection.immutable.List.foreach(List.scala:333)
137        at scala.tools.nsc.interpreter.shell.ILoop.interpretPreamble(ILoop.scala:924)
138        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$run$3(ILoop.scala:963)
139        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
140        at scala.tools.nsc.interpreter.shell.ILoop.echoOff(ILoop.scala:90)
141        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$run$2(ILoop.scala:963)
142        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
143        at scala.tools.nsc.interpreter.IMain.withSuppressedSettings(IMain.scala:1406)
144        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$run$1(ILoop.scala:954)
145        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
146        at scala.tools.nsc.interpreter.shell.ReplReporterImpl.withoutPrintingResults(Reporter.scala:64)
147        at scala.tools.nsc.interpreter.shell.ILoop.run(ILoop.scala:954)
148        at org.apache.spark.repl.Main$.doMain(Main.scala:84)
149        at org.apache.spark.repl.Main$.main(Main.scala:59)
150        at org.apache.spark.repl.Main.main(Main.scala)
151        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
152        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
153        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
154        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
155        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
156        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
157        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
158        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
159        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
160        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
161        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
162        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
16321/12/11 19:28:36 WARN MetricsSystem: Stopping a MetricsSystem that is not running
16421/12/11 19:28:36 ERROR Main: Failed to initialize Spark session.
165java.lang.reflect.InvocationTargetException
166        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
167        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
168        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
169        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
170        at org.apache.spark.executor.Executor.addReplClassLoaderIfNeeded(Executor.scala:909)
171        at org.apache.spark.executor.Executor.&lt;init&gt;(Executor.scala:160)
172        at org.apache.spark.scheduler.local.LocalEndpoint.&lt;init&gt;(LocalSchedulerBackend.scala:64)
173        at org.apache.spark.scheduler.local.LocalSchedulerBackend.start(LocalSchedulerBackend.scala:132)
174        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:220)
175        at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:581)
176        at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2690)
177        at org.apache.spark.sql.SparkSession$Builder.$anonfun$getOrCreate$2(SparkSession.scala:949)
178        at scala.Option.getOrElse(Option.scala:201)
179        at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:943)
180        at org.apache.spark.repl.Main$.createSparkSession(Main.scala:114)
181        at $line3.$read$$iw.&lt;init&gt;(&lt;console&gt;:5)
182        at $line3.$read.&lt;init&gt;(&lt;console&gt;:4)
183        at $line3.$read$.&lt;clinit&gt;(&lt;console&gt;)
184        at $line3.$eval$.$print$lzycompute(&lt;synthetic&gt;:6)
185        at $line3.$eval$.$print(&lt;synthetic&gt;:5)
186        at $line3.$eval.$print(&lt;synthetic&gt;)
187        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
188        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
189        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
190        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
191        at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:670)
192        at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1006)
193        at scala.tools.nsc.interpreter.IMain.$anonfun$doInterpret$1(IMain.scala:506)
194        at scala.reflect.internal.util.ScalaClassLoader.asContext(ScalaClassLoader.scala:36)
195        at scala.reflect.internal.util.ScalaClassLoader.asContext$(ScalaClassLoader.scala:116)
196        at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:43)
197        at scala.tools.nsc.interpreter.IMain.loadAndRunReq$1(IMain.scala:505)
198        at scala.tools.nsc.interpreter.IMain.$anonfun$doInterpret$3(IMain.scala:519)
199        at scala.tools.nsc.interpreter.IMain.doInterpret(IMain.scala:519)
200        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:503)
201        at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:501)
202        at scala.tools.nsc.interpreter.IMain.$anonfun$quietRun$1(IMain.scala:216)
203        at scala.tools.nsc.interpreter.shell.ReplReporterImpl.withoutPrintingResults(Reporter.scala:64)
204        at scala.tools.nsc.interpreter.IMain.quietRun(IMain.scala:216)
205        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$interpretPreamble$1(ILoop.scala:924)
206        at scala.collection.immutable.List.foreach(List.scala:333)
207        at scala.tools.nsc.interpreter.shell.ILoop.interpretPreamble(ILoop.scala:924)
208        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$run$3(ILoop.scala:963)
209        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
210        at scala.tools.nsc.interpreter.shell.ILoop.echoOff(ILoop.scala:90)
211        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$run$2(ILoop.scala:963)
212        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
213        at scala.tools.nsc.interpreter.IMain.withSuppressedSettings(IMain.scala:1406)
214        at scala.tools.nsc.interpreter.shell.ILoop.$anonfun$run$1(ILoop.scala:954)
215        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
216        at scala.tools.nsc.interpreter.shell.ReplReporterImpl.withoutPrintingResults(Reporter.scala:64)
217        at scala.tools.nsc.interpreter.shell.ILoop.run(ILoop.scala:954)
218        at org.apache.spark.repl.Main$.doMain(Main.scala:84)
219        at org.apache.spark.repl.Main$.main(Main.scala:59)
220        at org.apache.spark.repl.Main.main(Main.scala)
221        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
222        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
223        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
224        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
225        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
226        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:955)
227        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
228        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
229        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
230        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1043)
231        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1052)
232        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
233Caused by: java.net.URISyntaxException: Illegal character in path at index 42: spark://DESKTOP-JO73CF4.mshome.net:2103/C:\classes
234        at java.base/java.net.URI$Parser.fail(URI.java:2913)
235        at java.base/java.net.URI$Parser.checkChars(URI.java:3084)
236        at java.base/java.net.URI$Parser.parseHierarchical(URI.java:3166)
237        at java.base/java.net.URI$Parser.parse(URI.java:3114)
238        at java.base/java.net.URI.&lt;init&gt;(URI.java:600)
239        at org.apache.spark.repl.ExecutorClassLoader.&lt;init&gt;(ExecutorClassLoader.scala:57)
240        ... 67 more
24121/12/11 19:28:36 ERROR Utils: Uncaught exception in thread shutdown-hook-0
242java.lang.ExceptionInInitializerError
243        at org.apache.spark.executor.Executor.stop(Executor.scala:333)
244        at org.apache.spark.executor.Executor.$anonfun$stopHookReference$1(Executor.scala:76)
245        at org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:214)
246        at org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$2(ShutdownHookManager.scala:188)
247        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
248        at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:2019)
249        at org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$1(ShutdownHookManager.scala:188)
250        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
251        at scala.util.Try$.apply(Try.scala:210)
252        at org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:188)
253        at org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:178)
254        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
255        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
256        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
257        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
258        at java.base/java.lang.Thread.run(Thread.java:829)
259Caused by: java.lang.NullPointerException
260        at org.apache.spark.shuffle.ShuffleBlockPusher$.&lt;clinit&gt;(ShuffleBlockPusher.scala:465)
261        ... 16 more
26221/12/11 19:28:36 WARN ShutdownHookManager: ShutdownHook '' failed, java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
263java.util.concurrent.ExecutionException: java.lang.ExceptionInInitializerError
264        at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
265        at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:205)
266        at org.apache.hadoop.util.ShutdownHookManager.executeShutdown(ShutdownHookManager.java:124)
267        at org.apache.hadoop.util.ShutdownHookManager$1.run(ShutdownHookManager.java:95)
268Caused by: java.lang.ExceptionInInitializerError
269        at org.apache.spark.executor.Executor.stop(Executor.scala:333)
270        at org.apache.spark.executor.Executor.$anonfun$stopHookReference$1(Executor.scala:76)
271        at org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:214)
272        at org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$2(ShutdownHookManager.scala:188)
273        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
274        at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:2019)
275        at org.apache.spark.util.SparkShutdownHookManager.$anonfun$runAll$1(ShutdownHookManager.scala:188)
276        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
277        at scala.util.Try$.apply(Try.scala:210)
278        at org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:188)
279        at org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:178)
280        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
281        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
282        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
283        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
284        at java.base/java.lang.Thread.run(Thread.java:829)
285Caused by: java.lang.NullPointerException
286        at org.apache.spark.shuffle.ShuffleBlockPusher$.&lt;clinit&gt;(ShuffleBlockPusher.scala:465)
287        ... 16 more
288

As I can see it caused by Illegal character in path at index 42: spark://DESKTOP-JO73CF4.mshome.net:2103/C:\classes, but I don't understand what does it mean exactly and how to deal with that

How can I solve this problem?

I use Spark 3.2.0 Pre-built for Apache Hadoop 3.3 and later (Scala 2.13)

JAVA_HOME, HADOOP_HOME, SPARK_HOME path variables are set.

ANSWER

Answered 2022-Jan-07 at 15:11

i face the same problem, i think Spark 3.2 is the problem itself

switched to Spark 3.1.2, it works fine

Source https://stackoverflow.com/questions/70317481

QUESTION

ImportError: cannot import name 'safe_str_cmp' from 'werkzeug.security'

Asked 2022-Mar-28 at 21:19

any ideas why this error?

my project was working fine, i copied it to an external drive and onto my laptop to work on the road, it worked fine. i copied back to my desktop and had a load of issues with invalid interpreters etc, so i made a new project and copied just the scripts in, made a new requirements.txt and installed all the packages, but when i run i get this error

1Traceback (most recent call last):
2  File &quot;E:\Dev\spot_new\flask_blog\run.py&quot;, line 1, in &lt;module&gt;
3    from flaskblog import app
4  File &quot;E:\Dev\spot_new\flask_blog\flaskblog\__init__.py&quot;, line 3, in &lt;module&gt;
5    from flask_bcrypt import Bcrypt
6  File &quot;E:\Dev\spot_new\venv\lib\site-packages\flask_bcrypt.py&quot;, line 21, in &lt;module&gt;
7    from werkzeug.security import safe_str_cmp
8ImportError: cannot import name 'safe_str_cmp' from 'werkzeug.security' (E:\Dev\spot_new\venv\lib\site-packages\werkzeug\security.py)
9

i've tried uninstalling python, anaconda, pycharm, deleting every reg key and environment variable i can find that looks pythonic, reinstalling all from scratch but still no dice.

any input greatly appreciated!

thanks

ANSWER

Answered 2022-Mar-28 at 21:19

Werkzeug released v2.1.0 today, removing werkzeug.security.safe_str_cmp.

You can probably resolve this issue by pinning Werkzeug~=2.0.0 in your requirements.txt file (or similar).

Source https://stackoverflow.com/questions/71652965

QUESTION

What issue could I have in Gradle managed device setup?

Asked 2022-Mar-07 at 23:47

There was introduced a new feature Gradle managed devices (see for example here: https://developer.android.com/studio/preview/features?hl=fr)

The setup seems to be pretty straightforward, just copy a few lines to the module level build.gradle file and everything should work.

Sadly it is not the case for me and I strive for some advice, please. The code is red and the script doesn't succeed. See my build.gradle.kts file:

The underlined ManagedVirtualDevice shows the following error:

My Android studio version is Android Studio Bumblebee | 2021.1.1 Canary 11 Build #AI-211.7628.21.2111.7676841, built on August 26, 2021.

Syncing Gradle shows this:

1org.gradle.internal.exceptions.LocationAwareException: Build file '/*****/app/build.gradle.kts' line: 112
2Script compilation errors:
3
4  Line 112:             pixel2api29 (com.android.build.api.dsl.ManagedVirtualDevice) {
5                        ^ Unresolved reference: pixel2api29
6
7  Line 112:             pixel2api29 (com.android.build.api.dsl.ManagedVirtualDevice) {
8                                                               ^ Classifier 'ManagedVirtualDevice' does not have a companion object, and thus must be initialized here
9
10  Line 114:                 device = &quot;Pixel 2&quot;
11                            ^ Unresolved reference: device
12
13  Line 115:                 apiLevel = 29
14                            ^ Unresolved reference: apiLevel
15
16  Line 117:                 systemImageSource = &quot;google&quot;
17                            ^ Unresolved reference: systemImageSource
18
19  Line 118:                 abi = &quot;x86&quot;
20                            ^ Unresolved reference: abi
21
226 errors
23    at org.gradle.kotlin.dsl.execution.Interpreter$ProgramHost$compileSecondStageOf$cacheDir$1.invoke(Interpreter.kt:666)
24    at org.gradle.kotlin.dsl.execution.Interpreter$ProgramHost$compileSecondStageOf$cacheDir$1.invoke(Interpreter.kt:387)
25    at org.gradle.kotlin.dsl.provider.CompileKotlinScript.execute(KotlinScriptEvaluator.kt:375)
26    at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:89)
27    at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:40)
28    at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:53)
29    at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:50)
30    at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:200)
31    at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:195)
32    at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75)
33    ... 
34

Which

ANSWER

Answered 2021-Oct-15 at 11:43

Just ran into the same issue - you need to instantiate a ManagedVirtualDevice object and configure it, before adding it to your devices list:

1org.gradle.internal.exceptions.LocationAwareException: Build file '/*****/app/build.gradle.kts' line: 112
2Script compilation errors:
3
4  Line 112:             pixel2api29 (com.android.build.api.dsl.ManagedVirtualDevice) {
5                        ^ Unresolved reference: pixel2api29
6
7  Line 112:             pixel2api29 (com.android.build.api.dsl.ManagedVirtualDevice) {
8                                                               ^ Classifier 'ManagedVirtualDevice' does not have a companion object, and thus must be initialized here
9
10  Line 114:                 device = &quot;Pixel 2&quot;
11                            ^ Unresolved reference: device
12
13  Line 115:                 apiLevel = 29
14                            ^ Unresolved reference: apiLevel
15
16  Line 117:                 systemImageSource = &quot;google&quot;
17                            ^ Unresolved reference: systemImageSource
18
19  Line 118:                 abi = &quot;x86&quot;
20                            ^ Unresolved reference: abi
21
226 errors
23    at org.gradle.kotlin.dsl.execution.Interpreter$ProgramHost$compileSecondStageOf$cacheDir$1.invoke(Interpreter.kt:666)
24    at org.gradle.kotlin.dsl.execution.Interpreter$ProgramHost$compileSecondStageOf$cacheDir$1.invoke(Interpreter.kt:387)
25    at org.gradle.kotlin.dsl.provider.CompileKotlinScript.execute(KotlinScriptEvaluator.kt:375)
26    at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:89)
27    at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:40)
28    at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:53)
29    at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:50)
30    at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:200)
31    at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:195)
32    at org.gradle.internal.operations.DefaultBuildOperationRunner$3.execute(DefaultBuildOperationRunner.java:75)
33    ... 
34import com.android.build.gradle.internal.dsl.ManagedVirtualDevice
35
36testOptions {
37    devices {
38        add(
39            ManagedVirtualDevice(&quot;pixel2api30&quot;).apply {
40                device = &quot;Pixel 2&quot;
41                apiLevel = 30
42                systemImageSource = &quot;aosp-atd&quot;
43                abi = &quot;x86&quot;
44            }
45        )
46    }
47}
48

Note that the import is subtly different from what Google's documentation says - in their example they're passing the interface, not an implementation.

Source https://stackoverflow.com/questions/69159985

QUESTION

How to compose Free Monads

Asked 2022-Feb-20 at 05:20

1data Console a
2  = PutStrLn String a
3  | GetLine (String -&gt; a)
4  deriving (Functor)
5
6type ConsoleM = Free Console
7
8runConsole :: Console (IO a) -&gt; IO a
9runConsole cmd =
10  case cmd of
11    (PutStrLn s next) -&gt; do
12      putStrLn s
13      next
14    (GetLine nextF) -&gt; do
15      s &lt;- getLine
16      nextF s
17
18runConsoleM :: ConsoleM a -&gt; IO a
19runConsoleM = iterM runConsole
20
21consolePutStrLn :: String -&gt; ConsoleM ()
22consolePutStrLn str = liftF $ PutStrLn str ()
23consoleGetLine :: ConsoleM String
24consoleGetLine = liftF $ GetLine id
25
26
27data File a
28  = ReadFile FilePath (String -&gt; a)
29  | WriteFile FilePath String a
30  deriving (Functor)
31
32type FileM = Free File
33
34runFile :: File (MaybeT IO a) -&gt; MaybeT IO a
35runFile cmd = case cmd of
36  ReadFile path next -&gt; do
37    fileData &lt;- safeReadFile path
38    next fileData
39  WriteFile path fileData next -&gt; do
40    safeWriteFile path fileData
41    next
42
43runFileM :: FileM a -&gt; MaybeT IO a
44runFileM = iterM runFile
45
46rightToMaybe :: Either a b -&gt; Maybe b
47rightToMaybe = either (const Nothing) Just
48
49safeReadFile :: FilePath -&gt; MaybeT IO String
50safeReadFile path =
51  MaybeT $ rightToMaybe &lt;$&gt; (try $ readFile path :: IO (Either IOException String))
52
53safeWriteFile :: FilePath -&gt; String -&gt; MaybeT IO ()
54safeWriteFile path fileData =
55  MaybeT $ rightToMaybe &lt;$&gt; (try $ writeFile path fileData :: IO (Either IOException ()))
56
57fileReadFile :: FilePath -&gt; FileM String
58fileReadFile path = liftF $ ReadFile path id
59fileWriteFile :: FilePath -&gt; String -&gt; FileM ()
60fileWriteFile path fileData = liftF $ WriteFile path fileData ()
61
62
63data Program a = File (File a) | Console (Console a)
64  deriving (Functor)
65type ProgramM = Free Program
66
67runProgram :: Program (MaybeT IO a) -&gt; MaybeT IO a
68runProgram cmd = case cmd of
69  File cmd' -&gt;
70    runFile cmd'
71  Console cmd' -&gt;
72    -- ????
73
74runProgramM :: ProgramM a -&gt; MaybeT IO a
75runProgramM = iterM runProgram
76

I want to compose two free monads ConsoleM and FileM. So, I made composed functor Program. Then I started to write interpreter functrion runProgram, but I cannot define the function. Because runConsole and MaybeT IO a types are not matched. How can I lift runConsole function runConsole :: Console (IO a) -> IO a to have type Console (MaybeT IO a) -> MaybeT IO a?

(I want to implement this program with Free monads for practice, not Eff monad.)

ANSWER

Answered 2022-Feb-20 at 05:20

Now you have cmd' of type Console (MaybeT IO a), and want to pass it to a function taking Console (IO a). The first thing you can do is to run the MaybeT monad inside Console and get Console (IO (Maybe a)). You can do this by fmapping runMaybeT.

Once you got Console (IO (Maybe a)), you can pass it to runConsole and get IO (Maybe a). Now, you can lift it to MaybeT IO a using MaybeT.

So it'll be something like this.

1data Console a
2  = PutStrLn String a
3  | GetLine (String -&gt; a)
4  deriving (Functor)
5
6type ConsoleM = Free Console
7
8runConsole :: Console (IO a) -&gt; IO a
9runConsole cmd =
10  case cmd of
11    (PutStrLn s next) -&gt; do
12      putStrLn s
13      next
14    (GetLine nextF) -&gt; do
15      s &lt;- getLine
16      nextF s
17
18runConsoleM :: ConsoleM a -&gt; IO a
19runConsoleM = iterM runConsole
20
21consolePutStrLn :: String -&gt; ConsoleM ()
22consolePutStrLn str = liftF $ PutStrLn str ()
23consoleGetLine :: ConsoleM String
24consoleGetLine = liftF $ GetLine id
25
26
27data File a
28  = ReadFile FilePath (String -&gt; a)
29  | WriteFile FilePath String a
30  deriving (Functor)
31
32type FileM = Free File
33
34runFile :: File (MaybeT IO a) -&gt; MaybeT IO a
35runFile cmd = case cmd of
36  ReadFile path next -&gt; do
37    fileData &lt;- safeReadFile path
38    next fileData
39  WriteFile path fileData next -&gt; do
40    safeWriteFile path fileData
41    next
42
43runFileM :: FileM a -&gt; MaybeT IO a
44runFileM = iterM runFile
45
46rightToMaybe :: Either a b -&gt; Maybe b
47rightToMaybe = either (const Nothing) Just
48
49safeReadFile :: FilePath -&gt; MaybeT IO String
50safeReadFile path =
51  MaybeT $ rightToMaybe &lt;$&gt; (try $ readFile path :: IO (Either IOException String))
52
53safeWriteFile :: FilePath -&gt; String -&gt; MaybeT IO ()
54safeWriteFile path fileData =
55  MaybeT $ rightToMaybe &lt;$&gt; (try $ writeFile path fileData :: IO (Either IOException ()))
56
57fileReadFile :: FilePath -&gt; FileM String
58fileReadFile path = liftF $ ReadFile path id
59fileWriteFile :: FilePath -&gt; String -&gt; FileM ()
60fileWriteFile path fileData = liftF $ WriteFile path fileData ()
61
62
63data Program a = File (File a) | Console (Console a)
64  deriving (Functor)
65type ProgramM = Free Program
66
67runProgram :: Program (MaybeT IO a) -&gt; MaybeT IO a
68runProgram cmd = case cmd of
69  File cmd' -&gt;
70    runFile cmd'
71  Console cmd' -&gt;
72    -- ????
73
74runProgramM :: ProgramM a -&gt; MaybeT IO a
75runProgramM = iterM runProgram
76runProgram :: Program (MaybeT IO a) -&gt; MaybeT IO a
77runProgram cmd = case cmd of
78  File cmd' -&gt;
79    runFile cmd'
80  Console cmd' -&gt;
81    MaybeT $ runConsole $ fmap runMaybeT cmd'
82

Source https://stackoverflow.com/questions/71190849

QUESTION

Running one python script within another script using subprocess

Asked 2022-Feb-16 at 09:06

I am working on a script to walk over a directory, and convert all the python2 files to python3. There is a utitliy (2to3.py) to acheive that. ( I am using python2.7 interpreter)

I have the following code:

1import os
2import subprocess
3import pathlib
4
5APP_FOLDER = 'C:/Users/XXXX/Test/'
6
7for dirpath, dirnames, filenames in os.walk(APP_FOLDER):
8    for inputFile in filenames:
9        if pathlib.Path(inputFile).suffix == &quot;.py&quot;:
10            file_path = os.path.join(dirpath, inputFile)
11            with open(file_path) as f:
12                num_lines = len(f.readlines())
13            print dirpath
14            print inputFile
15            print &quot;lines of code: &quot;, num_lines
16            print &quot;converting to python 3&quot;
17            cmd =&quot;py C:\Python27\Tools\Scripts\\2to3.py &quot;+inputFile+&quot; -w&quot;
18            p1=subprocess.Popen(cmd,shell=True)
19            p1.wait()
20            if p1.returncode ==0:
21                print &quot;Success&quot;
22            else:
23                print &quot;failure&quot;
24
25

At the path mentioned, I have a subdirectory named "FolderA" and within that there is a simple python file having the syntax of python2 and a division operation.

I receive an error as below:

1import os
2import subprocess
3import pathlib
4
5APP_FOLDER = 'C:/Users/XXXX/Test/'
6
7for dirpath, dirnames, filenames in os.walk(APP_FOLDER):
8    for inputFile in filenames:
9        if pathlib.Path(inputFile).suffix == &quot;.py&quot;:
10            file_path = os.path.join(dirpath, inputFile)
11            with open(file_path) as f:
12                num_lines = len(f.readlines())
13            print dirpath
14            print inputFile
15            print &quot;lines of code: &quot;, num_lines
16            print &quot;converting to python 3&quot;
17            cmd =&quot;py C:\Python27\Tools\Scripts\\2to3.py &quot;+inputFile+&quot; -w&quot;
18            p1=subprocess.Popen(cmd,shell=True)
19            p1.wait()
20            if p1.returncode ==0:
21                print &quot;Success&quot;
22            else:
23                print &quot;failure&quot;
24
25C:/Users/XXXX/Test/FolderA
26Sample.py
27lines of code:  7
28converting to python 3
29RefactoringTool: Skipping optional fixer: buffer
30RefactoringTool: Skipping optional fixer: idioms
31RefactoringTool: Skipping optional fixer: set_literal
32RefactoringTool: Skipping optional fixer: ws_comma
33RefactoringTool: Can't open Sample.py: [Errno 2] No such file or directory: 'Sample.py'
34RefactoringTool: No files need to be modified.
35RefactoringTool: There was 1 error:
36RefactoringTool: Can't open Sample.py: [Errno 2] No such file or directory: 'Sample.py'
37failure
38-------------------*************--------------------
39

What am I doing wrong here?

ANSWER

Answered 2022-Feb-16 at 09:01

Try using, cmd ="py C:\Python27\Tools\Scripts\\2to3.py "+file_path+" -w"

Source https://stackoverflow.com/questions/71138693

QUESTION

How is CPython implemented?

Asked 2022-Feb-10 at 02:37

So I lately came across an explanation for Python's interpreter and compiler (CPython specifically).

Please correct me if I'm wrong. I just want to be sure I understand these specific concepts.

So CPython gets both compiled (to bytecode) and then interpreted (in the PVM)? And what does the PVM do exactly? Does it read the bytecode line by line, and translate each one to binary instructions that can be executed on a specific computer? Does this mean that a computer based on an Intel processor needs a different PVM from an AMD-based computer?

ANSWER

Answered 2022-Feb-09 at 13:49

Yes, CPython is compiled to bytecode which is then executed by the virtual machine.
The virtual machine executes instructions one-by-one. It's written in C (but you can write it in another language) and looks like a huge if/else statement like "if the current instruction is this, do this; if the instruction is this, do another thing", and so on. Instructions aren't translated to binary - that's why it's called an interpreter.
1. You can find the list of instructions here: https://docs.python.org/3.10/library/dis.html#python-bytecode-instructions
2. The implementation of the VM is available here: https://github.com/python/cpython/blob/f71a69aa9209cf67cc1060051b147d6afa379bba/Python/ceval.c#L1718
Bytecode doesn't have a concept of "line": it's just a stream of bytes. The interpreter can read one byte at a time and use another if/else statement to decide what instruction it's looking at. For example:

1curr_byte = read_byte()
2if curr_byte == 0x00:
3   # Parse instruction with no arguments
4   curr_instruction = DO_THING_A;
5   args = NULL;
6elif curr_byte == 0x01:
7   another_byte = read_byte()
8   if another_byte == 0x00:
9      # Parse a two-byte instruction
10      curr_instruction = DO_THING_B;
11      args = NULL;
12   else:
13      # Parse a one-byte instruction
14      # with one argument
15      curr_instruction = DO_THING_C;
16      args = another_byte &gt;&gt; 1; # or whatever
17elif curr_byte == ...:
18   ... # go on and on and on
19

The entire point of bytecode is that it can be executed by another program (the interpreter, or virtual machine) on almost any hardware. For example, in order to get CPython running on new hardware, you'll need a C toolchain (compiler, linker, assembler etc) for this hardware and a bunch of functions that Python can call to do low-level stuff (allocate memory, output text, do networking etc). Once you have that, write C code that can execute the bytecode - and that's it.

Source https://stackoverflow.com/questions/71050098

QUESTION

Stricter than strict mode?

Asked 2022-Jan-31 at 04:52

I recently produced a stupid bug:

1&quot;use strict&quot;;
2
3function doSomething() {
4    let testObject = {a: &quot;foo&quot;, b: &quot;bar&quot;, parent: &quot;bla&quot;};
5
6    if (parent in testObject) {
7        console.log(&quot;has a parent&quot;)
8    }
9    else {
10        console.log(&quot;does not have a parent&quot;)
11    }
12}
13
14doSomething();
15

Due to the missing quotes around the literal parent, the interpreter accessed window.parent and there was no ReferenceError as there would have been had I written a in testObject.

Obviously, JavaScript could not know that I my intention was not to access window.parent and thus could not have thrown an error. But I wonder whether there is some sort of debugging mode that would output a warning to the console in such cases, something along the line: "parent is not defined in this scope, accessing the global variable instead".

ANSWER

Answered 2022-Jan-30 at 10:53

No, JavaScript doesn't have a "stricter" mode that would have warned you of this. Some linters might (though ESLint doesn't seem to, at least with the demo page's default settings).

TypeScript would have, though (example), since window.parent isn't a string or Symbol, so doesn't make sense as the left-hand operand of in. Adopting TypeScript has costs, of course, but it does have benefits like this.

Source https://stackoverflow.com/questions/70913916

QUESTION

How to stop PyCharm's break/stop/halt feature on handled exceptions (i.e. only break on python unhandled exceptions)?

Asked 2022-Jan-26 at 00:10

I have that PyCharm is halting on all my exceptions, even the ones I am handling in a try except block. I do not want it to break there - I am handling and perhaps expecting an error. But every other exception I do want it to halt and suspend execution (e.g. so that I have the program state and debug it).

How does one do that?

I tried going into the python exception breakpoint option but I didn't see an option like "break only on unhandled exceptions" e.g as suggested by these:

note this is my current state, note how it stopped in my try block... :(

crossposted: https://intellij-support.jetbrains.com/hc/en-us/community/posts/4415666598546-How-to-stop-PyCharm-s-break-stop-halt-feature-on-handled-exceptions-i-e-only-break-on-python-unhandled-exceptions-

I tried:

1In your link here intellij-support.jetbrains.com/hc/en-us/community/posts/… the poster Okke said they solved this issue adding --pdb to the 'addition arguments', which someone later said they probably meant interpreter options.
2

but didn't work got error:

1In your link here intellij-support.jetbrains.com/hc/en-us/community/posts/… the poster Okke said they solved this issue adding --pdb to the 'addition arguments', which someone later said they probably meant interpreter options.
2/Users/brandomiranda/opt/anaconda3/envs/meta_learning/bin/python --pdb /Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevd.py --cmd-line --multiproc --qt-support=auto --client 127.0.0.1 --port 58378 --file /Users/brandomiranda/ultimate-utils/tutorials_for_myself/try_catch_pycharm_issues/try_catch_with_pickle.py
3unknown option --pdb
4usage: /Users/brandomiranda/opt/anaconda3/envs/meta_learning/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ...
5Try `python -h' for more information.
6
7Process finished with exit code 2
8

ANSWER

Answered 2022-Jan-25 at 23:49

I think it is already working actually, but you are in fact not catching the correct error. In your code you have:

1In your link here intellij-support.jetbrains.com/hc/en-us/community/posts/… the poster Okke said they solved this issue adding --pdb to the 'addition arguments', which someone later said they probably meant interpreter options.
2/Users/brandomiranda/opt/anaconda3/envs/meta_learning/bin/python --pdb /Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevd.py --cmd-line --multiproc --qt-support=auto --client 127.0.0.1 --port 58378 --file /Users/brandomiranda/ultimate-utils/tutorials_for_myself/try_catch_pycharm_issues/try_catch_with_pickle.py
3unknown option --pdb
4usage: /Users/brandomiranda/opt/anaconda3/envs/meta_learning/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ...
5Try `python -h' for more information.
6
7Process finished with exit code 2
8try:
9    pickle.dumps(obj)
10except pickle.PicklingError:
11    return False
12

But the error thrown is AttributeError. So to avoid that you need something like this:

1In your link here intellij-support.jetbrains.com/hc/en-us/community/posts/… the poster Okke said they solved this issue adding --pdb to the 'addition arguments', which someone later said they probably meant interpreter options.
2/Users/brandomiranda/opt/anaconda3/envs/meta_learning/bin/python --pdb /Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevd.py --cmd-line --multiproc --qt-support=auto --client 127.0.0.1 --port 58378 --file /Users/brandomiranda/ultimate-utils/tutorials_for_myself/try_catch_pycharm_issues/try_catch_with_pickle.py
3unknown option --pdb
4usage: /Users/brandomiranda/opt/anaconda3/envs/meta_learning/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ...
5Try `python -h' for more information.
6
7Process finished with exit code 2
8try:
9    pickle.dumps(obj)
10except pickle.PicklingError:
11    return False
12try:
13    pickle.dumps(obj)
14except (pickle.PicklingError, AttributeError):
15    return False
16

Source https://stackoverflow.com/questions/70761481

QUESTION

Wrong PHP Version/Executable in VSCode terminal but works perfectly in Mac terminal

Asked 2021-Dec-30 at 12:35

I just updated my Mac M1 to Big Sur 11.5.2 and something in VSCode seems to have broken. I am unable to use the latest home-brew php which is installed.

In VSCode its pointing to /usr/bin/php which is Macs built in php, that's not the one im using with home-brew. I tried everything and changed the path but still the same thing.

I checked the one similar question to mine and all it suggests is to use Homebrew which I already am doing so Im not sure what I am doing wrong here.

I am running PHPUnit tests in the VSCode terminal and I am getting the following error:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6

However when I run the same thing in the Mac terminal by going to the same folder it works perfectly:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6/Users/themyth/app/Sites/MapFramework/Map -&gt; ./vendor/bin/phpunit tests                                            
7PHPUnit 9.5.8 by Sebastian Bergmann and contributors.
8
9Runtime:       PHP 8.0.9
10Configuration: /Users/themyth/App/Sites/MapFramework/Map/phpunit.xml
11
12...R                                                                4 / 4 (100%)
13
14Time: 00:00.004, Memory: 6.00 MB
15
16There was 1 risky test:
17
181) tests\map\core\exception\MapExceptionTest::testDisplayMethodShowsBasicStaticHtml
19This test did not perform any assertions
20
21/Users/themyth/App/Sites/MapFramework/Map/tests/map/core/exception/MapExceptionTest.php:16
22
23OK, but incomplete, skipped, or risky tests!
24Tests: 4, Assertions: 4, Risky: 1.
25/Users/themyth/app/Sites/MapFramework/Map -&gt; 
26

When I do which php in both terminals I get a different result:

In Mac terminal:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6/Users/themyth/app/Sites/MapFramework/Map -&gt; ./vendor/bin/phpunit tests                                            
7PHPUnit 9.5.8 by Sebastian Bergmann and contributors.
8
9Runtime:       PHP 8.0.9
10Configuration: /Users/themyth/App/Sites/MapFramework/Map/phpunit.xml
11
12...R                                                                4 / 4 (100%)
13
14Time: 00:00.004, Memory: 6.00 MB
15
16There was 1 risky test:
17
181) tests\map\core\exception\MapExceptionTest::testDisplayMethodShowsBasicStaticHtml
19This test did not perform any assertions
20
21/Users/themyth/App/Sites/MapFramework/Map/tests/map/core/exception/MapExceptionTest.php:16
22
23OK, but incomplete, skipped, or risky tests!
24Tests: 4, Assertions: 4, Risky: 1.
25/Users/themyth/app/Sites/MapFramework/Map -&gt; 
26/Users/themyth/app/Sites/MapFramework/Map -&gt; which php
27/opt/homebrew/bin/php
28

In VSCode terminal:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6/Users/themyth/app/Sites/MapFramework/Map -&gt; ./vendor/bin/phpunit tests                                            
7PHPUnit 9.5.8 by Sebastian Bergmann and contributors.
8
9Runtime:       PHP 8.0.9
10Configuration: /Users/themyth/App/Sites/MapFramework/Map/phpunit.xml
11
12...R                                                                4 / 4 (100%)
13
14Time: 00:00.004, Memory: 6.00 MB
15
16There was 1 risky test:
17
181) tests\map\core\exception\MapExceptionTest::testDisplayMethodShowsBasicStaticHtml
19This test did not perform any assertions
20
21/Users/themyth/App/Sites/MapFramework/Map/tests/map/core/exception/MapExceptionTest.php:16
22
23OK, but incomplete, skipped, or risky tests!
24Tests: 4, Assertions: 4, Risky: 1.
25/Users/themyth/app/Sites/MapFramework/Map -&gt; 
26/Users/themyth/app/Sites/MapFramework/Map -&gt; which php
27/opt/homebrew/bin/php
28/Users/themyth/App/Sites/MapFramework/map -&gt; which php
29/usr/bin/php
30/Users/themyth/App/Sites/MapFramework/map -&gt; 
31

How can I point VSCode to the right version of PHP? I don't think this happened before doing the Big Sur update and I am not sure what to edit.

I tried to open settings.json but I can't find any info about this and I am not sure how to edit this.

Any advice would be appreciated.

Edit 1:

I think this issue happened since I installed PHPIntellisense on VSCode but can't be sure, what I do know is that it was working before. I dont know how to configure VSCode to point to the home-brew PHP which is already installed and working perfectly in the regular terminal

Edit 2

I tried to edit settings.json and it made no difference:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6/Users/themyth/app/Sites/MapFramework/Map -&gt; ./vendor/bin/phpunit tests                                            
7PHPUnit 9.5.8 by Sebastian Bergmann and contributors.
8
9Runtime:       PHP 8.0.9
10Configuration: /Users/themyth/App/Sites/MapFramework/Map/phpunit.xml
11
12...R                                                                4 / 4 (100%)
13
14Time: 00:00.004, Memory: 6.00 MB
15
16There was 1 risky test:
17
181) tests\map\core\exception\MapExceptionTest::testDisplayMethodShowsBasicStaticHtml
19This test did not perform any assertions
20
21/Users/themyth/App/Sites/MapFramework/Map/tests/map/core/exception/MapExceptionTest.php:16
22
23OK, but incomplete, skipped, or risky tests!
24Tests: 4, Assertions: 4, Risky: 1.
25/Users/themyth/app/Sites/MapFramework/Map -&gt; 
26/Users/themyth/app/Sites/MapFramework/Map -&gt; which php
27/opt/homebrew/bin/php
28/Users/themyth/App/Sites/MapFramework/map -&gt; which php
29/usr/bin/php
30/Users/themyth/App/Sites/MapFramework/map -&gt; 
31{
32    &quot;workbench.colorTheme&quot;: &quot;Monokai Dimmed&quot;,
33    &quot;security.workspace.trust.untrustedFiles&quot;: &quot;open&quot;,
34    &quot;redhat.telemetry.enabled&quot;: false,
35    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;,
36    &quot;php.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
37}
38

Edit 3

I have completely uninstalled VSCode and reinstalled everything. To start from fresh and the same problem happens. This is what the current settings.json file looks like:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6/Users/themyth/app/Sites/MapFramework/Map -&gt; ./vendor/bin/phpunit tests                                            
7PHPUnit 9.5.8 by Sebastian Bergmann and contributors.
8
9Runtime:       PHP 8.0.9
10Configuration: /Users/themyth/App/Sites/MapFramework/Map/phpunit.xml
11
12...R                                                                4 / 4 (100%)
13
14Time: 00:00.004, Memory: 6.00 MB
15
16There was 1 risky test:
17
181) tests\map\core\exception\MapExceptionTest::testDisplayMethodShowsBasicStaticHtml
19This test did not perform any assertions
20
21/Users/themyth/App/Sites/MapFramework/Map/tests/map/core/exception/MapExceptionTest.php:16
22
23OK, but incomplete, skipped, or risky tests!
24Tests: 4, Assertions: 4, Risky: 1.
25/Users/themyth/app/Sites/MapFramework/Map -&gt; 
26/Users/themyth/app/Sites/MapFramework/Map -&gt; which php
27/opt/homebrew/bin/php
28/Users/themyth/App/Sites/MapFramework/map -&gt; which php
29/usr/bin/php
30/Users/themyth/App/Sites/MapFramework/map -&gt; 
31{
32    &quot;workbench.colorTheme&quot;: &quot;Monokai Dimmed&quot;,
33    &quot;security.workspace.trust.untrustedFiles&quot;: &quot;open&quot;,
34    &quot;redhat.telemetry.enabled&quot;: false,
35    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;,
36    &quot;php.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
37}
38{
39    &quot;workbench.colorTheme&quot;: &quot;Default Dark+&quot;,
40    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
41}
42

When I do which php in the terminal in VSCode I always get the same thing /usr/bin/php

The same problem is continuing, Im not sure if this is because of the OSX update.

How could I configure the VSCode terminal to be exactly like the Mac terminal?

I also opened another project where I am using Symfony and I am trying to create a basic controller & that doesn't work too. I have edited the question to be more generic now:

php bin/console make:controller test I get the following error:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6/Users/themyth/app/Sites/MapFramework/Map -&gt; ./vendor/bin/phpunit tests                                            
7PHPUnit 9.5.8 by Sebastian Bergmann and contributors.
8
9Runtime:       PHP 8.0.9
10Configuration: /Users/themyth/App/Sites/MapFramework/Map/phpunit.xml
11
12...R                                                                4 / 4 (100%)
13
14Time: 00:00.004, Memory: 6.00 MB
15
16There was 1 risky test:
17
181) tests\map\core\exception\MapExceptionTest::testDisplayMethodShowsBasicStaticHtml
19This test did not perform any assertions
20
21/Users/themyth/App/Sites/MapFramework/Map/tests/map/core/exception/MapExceptionTest.php:16
22
23OK, but incomplete, skipped, or risky tests!
24Tests: 4, Assertions: 4, Risky: 1.
25/Users/themyth/app/Sites/MapFramework/Map -&gt; 
26/Users/themyth/app/Sites/MapFramework/Map -&gt; which php
27/opt/homebrew/bin/php
28/Users/themyth/App/Sites/MapFramework/map -&gt; which php
29/usr/bin/php
30/Users/themyth/App/Sites/MapFramework/map -&gt; 
31{
32    &quot;workbench.colorTheme&quot;: &quot;Monokai Dimmed&quot;,
33    &quot;security.workspace.trust.untrustedFiles&quot;: &quot;open&quot;,
34    &quot;redhat.telemetry.enabled&quot;: false,
35    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;,
36    &quot;php.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
37}
38{
39    &quot;workbench.colorTheme&quot;: &quot;Default Dark+&quot;,
40    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
41}
42Fatal error: Composer detected issues in your platform: Your Composer dependencies require a PHP version &quot;&gt;= 8.0.0&quot;. You are running 7.3.24-(to be removed in future macOS)
43

What is most frustrating is this seems so straight forward I can't understand what I need to do to fix this

ANSWER

Answered 2021-Aug-25 at 09:40

I got the same problem. Open your terminal and write this:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6/Users/themyth/app/Sites/MapFramework/Map -&gt; ./vendor/bin/phpunit tests                                            
7PHPUnit 9.5.8 by Sebastian Bergmann and contributors.
8
9Runtime:       PHP 8.0.9
10Configuration: /Users/themyth/App/Sites/MapFramework/Map/phpunit.xml
11
12...R                                                                4 / 4 (100%)
13
14Time: 00:00.004, Memory: 6.00 MB
15
16There was 1 risky test:
17
181) tests\map\core\exception\MapExceptionTest::testDisplayMethodShowsBasicStaticHtml
19This test did not perform any assertions
20
21/Users/themyth/App/Sites/MapFramework/Map/tests/map/core/exception/MapExceptionTest.php:16
22
23OK, but incomplete, skipped, or risky tests!
24Tests: 4, Assertions: 4, Risky: 1.
25/Users/themyth/app/Sites/MapFramework/Map -&gt; 
26/Users/themyth/app/Sites/MapFramework/Map -&gt; which php
27/opt/homebrew/bin/php
28/Users/themyth/App/Sites/MapFramework/map -&gt; which php
29/usr/bin/php
30/Users/themyth/App/Sites/MapFramework/map -&gt; 
31{
32    &quot;workbench.colorTheme&quot;: &quot;Monokai Dimmed&quot;,
33    &quot;security.workspace.trust.untrustedFiles&quot;: &quot;open&quot;,
34    &quot;redhat.telemetry.enabled&quot;: false,
35    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;,
36    &quot;php.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
37}
38{
39    &quot;workbench.colorTheme&quot;: &quot;Default Dark+&quot;,
40    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
41}
42Fatal error: Composer detected issues in your platform: Your Composer dependencies require a PHP version &quot;&gt;= 8.0.0&quot;. You are running 7.3.24-(to be removed in future macOS)
43nano ~/.zshrc
44

At the top of the file you have this:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6/Users/themyth/app/Sites/MapFramework/Map -&gt; ./vendor/bin/phpunit tests                                            
7PHPUnit 9.5.8 by Sebastian Bergmann and contributors.
8
9Runtime:       PHP 8.0.9
10Configuration: /Users/themyth/App/Sites/MapFramework/Map/phpunit.xml
11
12...R                                                                4 / 4 (100%)
13
14Time: 00:00.004, Memory: 6.00 MB
15
16There was 1 risky test:
17
181) tests\map\core\exception\MapExceptionTest::testDisplayMethodShowsBasicStaticHtml
19This test did not perform any assertions
20
21/Users/themyth/App/Sites/MapFramework/Map/tests/map/core/exception/MapExceptionTest.php:16
22
23OK, but incomplete, skipped, or risky tests!
24Tests: 4, Assertions: 4, Risky: 1.
25/Users/themyth/app/Sites/MapFramework/Map -&gt; 
26/Users/themyth/app/Sites/MapFramework/Map -&gt; which php
27/opt/homebrew/bin/php
28/Users/themyth/App/Sites/MapFramework/map -&gt; which php
29/usr/bin/php
30/Users/themyth/App/Sites/MapFramework/map -&gt; 
31{
32    &quot;workbench.colorTheme&quot;: &quot;Monokai Dimmed&quot;,
33    &quot;security.workspace.trust.untrustedFiles&quot;: &quot;open&quot;,
34    &quot;redhat.telemetry.enabled&quot;: false,
35    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;,
36    &quot;php.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
37}
38{
39    &quot;workbench.colorTheme&quot;: &quot;Default Dark+&quot;,
40    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
41}
42Fatal error: Composer detected issues in your platform: Your Composer dependencies require a PHP version &quot;&gt;= 8.0.0&quot;. You are running 7.3.24-(to be removed in future macOS)
43nano ~/.zshrc
44# If you come from bash you might have to change your $PATH.
45# export PATH=$HOME/bin:/usr/local/bin:$PATH
46

Put this line just under:

1/Users/themyth/App/Sites/MapFramework/map -&gt; ./vendor/bin/phpunit tests                           
2/usr/bin/php declares an invalid value for PHP_VERSION.
3This breaks fundamental functionality such as version_compare().
4Please use a different PHP interpreter.
5/Users/themyth/App/Sites/MapFramework/map -&gt;
6/Users/themyth/app/Sites/MapFramework/Map -&gt; ./vendor/bin/phpunit tests                                            
7PHPUnit 9.5.8 by Sebastian Bergmann and contributors.
8
9Runtime:       PHP 8.0.9
10Configuration: /Users/themyth/App/Sites/MapFramework/Map/phpunit.xml
11
12...R                                                                4 / 4 (100%)
13
14Time: 00:00.004, Memory: 6.00 MB
15
16There was 1 risky test:
17
181) tests\map\core\exception\MapExceptionTest::testDisplayMethodShowsBasicStaticHtml
19This test did not perform any assertions
20
21/Users/themyth/App/Sites/MapFramework/Map/tests/map/core/exception/MapExceptionTest.php:16
22
23OK, but incomplete, skipped, or risky tests!
24Tests: 4, Assertions: 4, Risky: 1.
25/Users/themyth/app/Sites/MapFramework/Map -&gt; 
26/Users/themyth/app/Sites/MapFramework/Map -&gt; which php
27/opt/homebrew/bin/php
28/Users/themyth/App/Sites/MapFramework/map -&gt; which php
29/usr/bin/php
30/Users/themyth/App/Sites/MapFramework/map -&gt; 
31{
32    &quot;workbench.colorTheme&quot;: &quot;Monokai Dimmed&quot;,
33    &quot;security.workspace.trust.untrustedFiles&quot;: &quot;open&quot;,
34    &quot;redhat.telemetry.enabled&quot;: false,
35    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;,
36    &quot;php.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
37}
38{
39    &quot;workbench.colorTheme&quot;: &quot;Default Dark+&quot;,
40    &quot;php.validate.executablePath&quot;: &quot;/opt/homebrew/bin/php&quot;
41}
42Fatal error: Composer detected issues in your platform: Your Composer dependencies require a PHP version &quot;&gt;= 8.0.0&quot;. You are running 7.3.24-(to be removed in future macOS)
43nano ~/.zshrc
44# If you come from bash you might have to change your $PATH.
45# export PATH=$HOME/bin:/usr/local/bin:$PATH
46export PATH=/opt/homebrew/opt/php@8.0/bin:$PATH
47

Save and close, restart your terminal and it will normally work.

NB: I write php@8.0 but you can do this with all versions you install with homebrew

Source https://stackoverflow.com/questions/68899144

QUESTION

Why does this Python code with threading have race conditions?

Asked 2021-Dec-27 at 17:33

This code creates a race condition:

1import threading
2
3ITERS = 100000
4x = [0]
5
6def worker():
7    for _ in range(ITERS):
8        x[0] += 1  # this line creates a race condition
9        # because it takes a value, increments and then writes
10        # some inrcements can be done together, and lost
11
12def main():
13    x[0] = 0  # you may use `global x` instead of this list trick too
14    t1 = threading.Thread(target=worker)
15    t2 = threading.Thread(target=worker)
16    t1.start()
17    t2.start()
18    t1.join()
19    t2.join()
20
21for i in range(5):
22    main()
23    print(f'iteration {i}. expected x = {ITERS*2}, got {x[0]}')
24

Output:

1import threading
2
3ITERS = 100000
4x = [0]
5
6def worker():
7    for _ in range(ITERS):
8        x[0] += 1  # this line creates a race condition
9        # because it takes a value, increments and then writes
10        # some inrcements can be done together, and lost
11
12def main():
13    x[0] = 0  # you may use `global x` instead of this list trick too
14    t1 = threading.Thread(target=worker)
15    t2 = threading.Thread(target=worker)
16    t1.start()
17    t2.start()
18    t1.join()
19    t2.join()
20
21for i in range(5):
22    main()
23    print(f'iteration {i}. expected x = {ITERS*2}, got {x[0]}')
24$ python3 test.py
25iteration 0. expected x = 200000, got 200000
26iteration 1. expected x = 200000, got 148115
27iteration 2. expected x = 200000, got 155071
28iteration 3. expected x = 200000, got 200000
29iteration 4. expected x = 200000, got 200000
30

Python3 version:

1import threading
2
3ITERS = 100000
4x = [0]
5
6def worker():
7    for _ in range(ITERS):
8        x[0] += 1  # this line creates a race condition
9        # because it takes a value, increments and then writes
10        # some inrcements can be done together, and lost
11
12def main():
13    x[0] = 0  # you may use `global x` instead of this list trick too
14    t1 = threading.Thread(target=worker)
15    t2 = threading.Thread(target=worker)
16    t1.start()
17    t2.start()
18    t1.join()
19    t2.join()
20
21for i in range(5):
22    main()
23    print(f'iteration {i}. expected x = {ITERS*2}, got {x[0]}')
24$ python3 test.py
25iteration 0. expected x = 200000, got 200000
26iteration 1. expected x = 200000, got 148115
27iteration 2. expected x = 200000, got 155071
28iteration 3. expected x = 200000, got 200000
29iteration 4. expected x = 200000, got 200000
30Python 3.9.7 (default, Sep 10 2021, 14:59:43) 
31[GCC 11.2.0] on linux
32

I thought GIL would prevent it and not allow two threads run together until they do something io-related or call a C library. At least this is what you may conclude from the docs.

Then, what does GIL actually do, and when do threads run in parallel?

ANSWER

Answered 2021-Dec-27 at 17:33

Reading the docs better, I think there's the answer:

The mechanism used by the CPython interpreter to assure that only one thread executes Python bytecode at a time. This simplifies the CPython implementation by making the object model (including critical built-in types such as dict) implicitly safe against concurrent access. Locking the entire interpreter makes it easier for the interpreter to be multi-threaded, at the expense of much of the parallelism afforded by multi-processor machines.

However, some extension modules, either standard or third-party, are designed so as to release the GIL when doing computationally-intensive tasks such as compression or hashing. Also, the GIL is always released when doing I/O.

I don't know the internals, but guess each line or block of this bytecode is executed alone, and other threads are waiting (which makes it slow). But some lines consist of multiple blocks, and aren't atomic.

Here's what you get if run dis.dis('x[0] += 1'):

1import threading
2
3ITERS = 100000
4x = [0]
5
6def worker():
7    for _ in range(ITERS):
8        x[0] += 1  # this line creates a race condition
9        # because it takes a value, increments and then writes
10        # some inrcements can be done together, and lost
11
12def main():
13    x[0] = 0  # you may use `global x` instead of this list trick too
14    t1 = threading.Thread(target=worker)
15    t2 = threading.Thread(target=worker)
16    t1.start()
17    t2.start()
18    t1.join()
19    t2.join()
20
21for i in range(5):
22    main()
23    print(f'iteration {i}. expected x = {ITERS*2}, got {x[0]}')
24$ python3 test.py
25iteration 0. expected x = 200000, got 200000
26iteration 1. expected x = 200000, got 148115
27iteration 2. expected x = 200000, got 155071
28iteration 3. expected x = 200000, got 200000
29iteration 4. expected x = 200000, got 200000
30Python 3.9.7 (default, Sep 10 2021, 14:59:43) 
31[GCC 11.2.0] on linux
32          0 LOAD_NAME                0 (x)
33          2 LOAD_CONST               0 (0)
34          4 DUP_TOP_TWO
35          6 BINARY_SUBSCR
36          8 LOAD_CONST               1 (1)
37         10 INPLACE_ADD
38         12 ROT_THREE
39         14 STORE_SUBSCR
40         16 LOAD_CONST               2 (None)
41         18 RETURN_VALUE
42

Some of these are executed in concurrent way, and make the race condition. So GIL only guarantees the internals of structures like list or dict won't be damaged.

Source https://stackoverflow.com/questions/70493438

Community Discussions contain sources that include Stack Exchange Network