bigtop | Manhattan plot in colorful virtual reality
kandi X-RAY | bigtop Summary
kandi X-RAY | bigtop Summary
BigTop demonstrates some of the potential of virtual reality for scientific visualization by providing an easy way for scientists and science enthusiasts to become immersed in their data. This provides several key advantages over more traditional 2D visualization methods, such as:. While BigTop is meant for examining large genomic data sets such as those found in GWAS studies, in reality it can be used to visualize any data set that contains genomic location, p-value, and a factor that can be quantified between 0 and 1 (usually frequency of an attribute's expression in a cohort).
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of bigtop
bigtop Key Features
bigtop Examples and Code Snippets
Community Discussions
Trending Discussions on bigtop
QUESTION
I am new to bigtop architecture, I would like to know
how does bigtop know the real build command to launch for this specific package
after ./gradlew {package}-rpm
, I assume there must be some kind of configs define the real build command. (The package
is a maven based project)
Thank you.
...ANSWER
Answered 2020-Apr-01 at 23:15I'm not familiar with Bigtop, but I am familiar with Gradle. See here for the Gradle task definition that you're referring to: https://github.com/apache/bigtop/blob/2d6f3dd7b7241aa2191c9ebc5a502a1415932464/packages.gradle#L460
The command that the task will execute is given under the exec
directive: rpmbuild
. command
is an array of arguments defined just above that directive. Most of its arguments are derived from the config
object, which is basically a nested map (like a JSON object) produced by Groovy's ConfigSlurper, which reads the input BOM file as if it were a Groovy file.
So:
- "Slurp" the BOM configuation into the
config
object - For each "component" defined within the
config
configuration, produce a set of tasks (${package}-rpm
and others) - When configuring the
${package}-rpm
task, use the BOM configuration to derive the command arguments using the logic provided within the task closure - Upon execution, run
rpmbuild
with the aforementioned command arguments
QUESTION
I have a Hive table tweets stored as text that I am trying to write to another table tweetsORC that is ORC. Both have the same structure:
...ANSWER
Answered 2017-Feb-10 at 09:07Instead of using Select * I list the fields by name and the error goes.
QUESTION
Can someone let me know how can I view all the nested dependencies of a package in Ubuntu? For example,
...ANSWER
Answered 2017-Dec-22 at 19:57Well, there is the apt-cache dotty
command, which will generate a graphviz representation of the package's dependencies. However, this is going to be less useful than you think: there are a lot of "core packages" that are going to be required by just about everything, and the resulting graphs will be quite large.
For example, the output of apt-cache dotty openssh-client
renders into this beast.
The dot syntax is relatively simple; you could probably parse that yourself to extract a subset of the information.
QUESTION
I'm using BigTop 1.2.0 the vagrant provisioner is running fine but when I try to use the Docker provisioner the following error is showing:
Command
$ ./docker-hadoop.sh -c 3
Error
Environment check... Check docker: Docker version 17.09.0-ce, build afdb6d4 Check docker-compose: docker-compose version 1.16.1, build 6d1ac219 Check ruby: ruby 2.4.2p198 (2017-09-14 revision 59899) [x64-mingw32] -e:1:in
': undefined method
[]' for nil:NilClass (NoMethodError) -e:1:in': undefined method
[]' for nil:NilClass (NoMethodError) The scale command is deprecated. Use the up command with the --scale flag instead. Creating 20171020215450r21863_bigtop_1 ... Creating 20171020215450r21863_bigtop_2 ... Creating 20171020215450r21863_bigtop_3 ...ERROR: for 20171020215450r21863_bigtop_2 Resource ID was not provided ERROR: for 20171020215450r21863_bigtop_3 Resource ID was not provided ERROR: for 20171020215450r21863_bigtop_1 Resource ID was not provided
Traceback (most recent call last):
File "docker-compose", line 6, in
File "compose\cli\main.py", line 68, in main
File "compose\cli\main.py", line 121, in perform_command
File "compose\cli\main.py", line 796, in scale
File "compose\service.py", line 257, in scale File "compose\service.py", line 444, in _execute_convergence_start File "compose\service.py", line 395, in _execute_convergence_create File "compose\parallel.py", line 70, in parallel_execute docker.errors.NullResource: Resource ID was not provided Failed to execute script docker-compose Docker container(s) startup failed!
Thanks in advance.
...ANSWER
Answered 2017-Dec-08 at 14:43It seems that you're running docker-hadoop.sh on windows with mingw32. The Docker Provisioner is tested to work on Mac and Amazon Linux for Bigtop releases just because of the resource limitation. So I'm not quite sure what's the story at Windows side ;)
However, from your error log:
undefined method[]' for nil:NilClass (NoMethodError)
I'm pretty sure that the issue is related to our ruby YAML parsing feature: https://github.com/apache/bigtop/blob/master/provisioner/docker/docker-hadoop.sh#L138
To this end, I'd suggest you to stick with Vagrant Provisioner on Windows. If you'd like to help us in making the feature compatible with Windows, feel free to reach out to dev@bigtop.apache.org
QUESTION
I have a docker image with USER bigtop
in the Dockerfile. I do:
ANSWER
Answered 2017-Oct-17 at 16:41The docker export
command will export the container filesystem (no Docker image or runtime metadata). See https://docs.docker.com/engine/reference/commandline/export/.
Essentially, when you export the filesystem this way, you lose any of the Docker specific instructions including USER
.
It is worth noting that you can export from images (not containers) and save the image and metadata in a tarball using docker save
: https://docs.docker.com/engine/reference/commandline/save/#usage.
QUESTION
I'm seeking some help, I have been tasked with standing up a Hadoop cluster at work. I have done single node stuff on laptops at home with the open source stack (I am trying to stick with the open source, Apache stack to avoid any licensing costs. Right now we have no interest in Cloudera or HortonWorks.).
I came across the Apache BigTop stack (1.2.0) and poked around in there. Right now I am still trying to wrap my head around what this provides (I have not found a reference to Hadoop/Spark versions, etc..). Could I get some help on the following:
What versions of Hadoop/Spark/other tools does the 1.2.0 version provide?
Is there a good reference on installing a full Hadoop/Spark cluster from scratch under RHEL 7? I have 12 servers, I plan on doing 2 namenodes and 10 datanodes. Is BigTop appropriate for this, or should I just install each package and configure manually?
I found the following:
Which looks promising, but its for CentOS 7, which I know is similar, but not exactly the same. Can someone suggest how I can modify this to work under RHEL 7? I found repos, but none for RHEL....
- The documentation seems pretty slim on the official Apache page, or maybe I'm just not looking in the right spot... Are there good links to references out there for a full cluster install?
Thanks to all who can help, I really appreciate it!
...ANSWER
Answered 2017-Oct-14 at 14:53What versions of Hadoop/Spark/other tools does the 1.2.0 version provide?
Checkout our doc for 1.2.0 release:
https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+1.2.0+Release
You'll get hadoop 2.7.3 and spark 2.1.0 out-of-the-box. We've provided installable artifacts on S3 for you to test out the functionality
https://www.apache.org/dist/bigtop/bigtop-1.2.0/repos/centos7/bigtop.repo
NOTE: we'll have a S3 migration effectively on 10/15, 2017. We'll have corresponding changes afterwards. If you'd like to try it out ASAP. Please change the baseurl to:
http://repos.bigtop.apache.org/releases/1.2.0/centos/7/x86_64
Is there a good reference on installing a full Hadoop/Spark cluster from scratch under RHEL 7? I have 12 servers, I plan on doing 2 namenodes and 10 datanodes. Is BigTop appropriate for this, or should I just install each package and configure manually?
RHEL and CentOS should be very much similar. I suggest:
- Try our CentOS packages directly on RHEL and see if that works. I've used Bigtop CentOS 6 packages on RHEL 6 in Production and it works like a charm.
- If above doesn't work, Bigtop is a fully open sourced solution for you to build up your own Hadoop Distribution. You can build the entire stack up against your desired Distro. from scratch. We've well crafted tools and dockerlized framework to support it. If you what to do so, raise your need at user@bigtop.apache.org mailing list. We'd be happy to help.
I found the following: https://cwiki.apache.org/confluence/display/BIGTOP/Bigtop+1.2.0+Release
Yes. You're looking for the right doc. And this is exactly what I've mentioned above: though it's for CentOS 7, you can try the repo on RHEL 7.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install bigtop
npm install to install the required libraries
npm run build to build and package the project's files
Navigate to the build directory and launch index.html to view
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page