tungsten | High performance physically based renderer in C++11 | Parser library

I would like to confirm some aspects that from reading all blogs and Databricks sources and the experts Holden, Warren et al, seem still poorly explained. I also note some unanswered questions out there in the net regarding this topic.

The following then:

• My understanding is that for RDD's we have the DAG Scheduler that creates the Stages in a simple manner.

• My understanding based on reading elsewhere to-date is that for DF's and DS's that we:

  • Use Catalyst instead of the DAG Scheduler. But I am not convinced.
  • Have Tungsten.

As DAG applies to DF's and DS's as well (obviously), I am left with 1 question - just to be sure:

• Is it Catalyst that creates the Stages as well?
  • This may seem a silly question, but I noted a question on Disable Spark Catalyst Optimizer here on SO. That would imply no.
  • In addition, as the Spark paradigm is Stage based (shuffle boundaries), it seems to me that deciding Stages is not a Catalyst thing.

Therefore my conclusion is that the DAG Scheduler is still used for Stages with DF's and DS's, but I am looking for confirmation.

Moreover, this picture implies that there is still a DAG Scheduler.

This picture from the Databricks 2019 summit seems in contrast to the statement found on a blog:

An important element helping Dataset to perform better is Catalyst Optimizer (CO), an internal query optimizer. It "translates" transformations used to build the Dataset to physical plan of execution. Thus, it's similar to DAG scheduler used to create physical plan of execution of RDD. ...

I see many unanswered questions on SO on the DAGs with DF's etc. and a lot of stuff is out-of-date as it RDD related. So, as a consequence I asked a round a few of my connection with Spark knowledge on this and noted they were remiss in providing a suitable answer.

so I've coded a questionnaire and I'm running into an issue. When a user clicks submit the questionnaire the results are displayed near the submit button I was hoping for them to be redirected to a new page (where results are displayed) by using the CSS hidden property, but I don't understand how I can do that and also for some reason after submitting there is an 'undefined' option displaying under the choices, how can I get rid of that? I'd really appreciate it if someone can help me solve my issues, thank you!

I linked the IDE for my project if that's easier: https://repl.it/@AS11RA/Forest-Firefighters-Website#start%20questionnaire.js

Heres the startquestionnaire.Js file:

so I've coded a questionnaire/survey for my website but for some reason when a user clicks start questionnaire the questions show but the choices aren't displayed. I'm really confused as to why this isn't working I would really appreciate if anyone can help me solve my problem, thank you!

Here's a link to my IDE as I figured it would be easier to go through it to identify any mistakes I'm making: https://repl.it/@AS11RA/Forest-Firefighters-Website#index.html

Here's the start questionnaire.js file:

I'm trying to build a really simple survey on Javascript but I keep getting this error "TypeError: startSurvey is not a function at HTMLButtonElement.onclick (/:2:54)" I would really appreciate if anybody can help me solve this error or provide any further feedback and suggestions.

Here's my HTML code:

I wanted to return the name of elements based on two conditions; even protons number and odd neutrons number. I've tried to print both tests and it turns out well. However, when I try to print the elements using 'and' logical, an error has occurred due to different broadcasting. I can't figure out how do I reshape it. Help me out.

The elements, protons and neutrons.

I've already converted elements, protons and neutrons into arrays.

The input;

In previous versions, there is a configuration called spark.shuffle.manage which is used to determine the type of shuffle algorithm in Spark. Since Spark 2.0, this configuration is deleted. The default shuffle algorithm is the sort-based. As my understanding, the Tungsten shuffle will be enabled only if all the requirements are satisfied. How can I know whether the current job uses the original sort-based shuffle or Tungsten sort?

Thank you very much.

I just started programming for my job and I am stuck on something. I looked online before but none of the answers seemed to work. I am using BeautifulSoup but Im open to using something else. Thank you so much!

I am trying to extract the names in

i am trying to read the password from file and passing into SAP connection. Since i have 50+ scripts using same password, where if password gets chnaged i no need to go in each script and change instead i can simply change in one file.

I thought that with the integration of project Tungesten, spark would automatically use off heap memory.

What for are spark.memory.offheap.size and spark.memory.offheap.enabled? Do I manually need to specify the amount of off heap memory for Tungsten here?