ambari | Apache Ambari simplifies provisioning managing

 by   apache Java Version: 2.0.0.0 License: Apache-2.0

kandi X-RAY | ambari Summary

kandi X-RAY | ambari Summary

ambari is a Java library typically used in Big Data, Hadoop applications. ambari has no bugs, it has build file available, it has a Permissive License and it has high support. However ambari has 12 vulnerabilities. You can download it from GitHub, Maven.

Apache Ambari is a tool for provisioning, managing, and monitoring Apache Hadoop clusters. Ambari consists of a set of RESTful APIs and a browser-based management interface.
Support
    Quality
      Security
        License
          Reuse

            kandi-support Support

              ambari has a highly active ecosystem.
              It has 1925 star(s) with 1593 fork(s). There are 206 watchers for this library.
              OutlinedDot
              It had no major release in the last 12 months.
              ambari has no issues reported. There are 124 open pull requests and 0 closed requests.
              OutlinedDot
              It has a negative sentiment in the developer community.
              The latest version of ambari is 2.0.0.0

            kandi-Quality Quality

              ambari has no bugs reported.

            kandi-Security Security

              ambari has 12 vulnerability issues reported (3 critical, 3 high, 5 medium, 1 low).

            kandi-License License

              ambari is licensed under the Apache-2.0 License. This license is Permissive.
              Permissive licenses have the least restrictions, and you can use them in most projects.

            kandi-Reuse Reuse

              ambari releases are not available. You will need to build from source code and install.
              Deployable package is available in Maven.
              Build file is available. You can build the component from source.
              Installation instructions are available. Examples and code snippets are not available.

            Top functions reviewed by kandi - BETA

            kandi has reviewed ambari and discovered the below as its top functions. This is intended to give you an instant insight into ambari implemented functionality, and help decide if they suit your requirements.
            • Perform stage creation .
            • Get a resource definition for a given type .
            • Executes the command .
            • Creates the next upgrade group .
            • Check all services and their configurations .
            • Add a custom command
            • Populates the specified alert definition entity with the specified map .
            • Updates service components .
            • Update the services in the request .
            • Initialize the prepared statements .
            Get all kandi verified functions for this library.

            ambari Key Features

            No Key Features are available at this moment for ambari.

            ambari Examples and Code Snippets

            Execute sql queries from shell script
            Lines of Code : 13dot img1License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            $ psql -U ambari -c "SELECT ... ;" ambari    # > result.file
            
            $ cat file.sql
            SELECT
            ... ;
            $ psql -U ambari -f file.sql ambari          # > result.file
            
            $ cat >> ~/.pgpass
            #hostnam
            Is it possible to send metrics to prometheus?
            Lines of Code : 49dot img2License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            apiVersion: v1
            kind: Endpoints
            metadata:
              name: ambari-metrics
              labels:
                app: ambari
            subsets:
            - addresses:
              - ip: "ambari-external-ip"
            ports:
            - name: metrics
              port: 9100
              protocol: TCP
            ---
            apiVersion: v1
            kind: Service
            metadata:
              n
            Ansible TypeError: must be string or buffer, not list
            Lines of Code : 15dot img3License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            - name: Provide Ambari cluster user role to users in file {{ cluster_user_group_file }}
              uri:
                  url: "http://{{ ansible_fqdn }}:8080/api/v1/clusters/{{ cluster_name }}/privileges"
                  method: POST
                  force_basic_auth: yes
                  u
            How to add host into the ambari-server cluster?
            Lines of Code : 29dot img4License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            #!/bin/bash
            # variables
            HOSTNAME=$(curl http://169.254.169.254/latest/meta-data/hostname)
            hostname $HOSTNAME
            USERNAME='username'
            PASSWORD='password'
            CLUSTER_NAME='mycluster'
            AMBARI_HOST='ip-172-166-11-52.ap-south-1.compute.internal'
            AMBARI
            copy iconCopy
            Ambari UI 
               -> Spark2 
                   -> Config
                      -> Advanced spark2-env
            
            How to retrieve the name node host name using Ambari?
            Lines of Code : 31dot img6License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            ! pip install --quiet python-ambariclient
            
            from future.standard_library import install_aliases
            install_aliases()
            from urllib.parse import urlparse
            
            import json
            vcap = json.load(open('./vcap.json'))
            
            USER         = v
            ConnectionError connecting to IBM Analytics Engine
            Lines of Code : 34dot img7License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            ...
            url = urlparse(AMBARI_URL)
            
            HOST = url.hostname
            PORT = url.port
            PROTOCOL = url.scheme  
            ...
            
            ambari = Ambari(HOST, ... , protocol=PROTOCOL)
            
            from future.standard_library import install_aliases
            install_aliases()
            
            Ambari-Update failed (Ambari 2.4 to 2.6) - Hadoop services won't start anymore
            Lines of Code : 16dot img8License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            # delete the new ambari repo file
            rm /etc/yum.repos.d/ambari.repo
            
            # download the old ambari repo file (for me version 2.4.2), as described in ambari installation guide (here for Centos 7)
            wget -nv http://public-repo-1.hortonworks.com/amba
            Ambari won't restart: DB check failed
            Lines of Code : 17dot img9License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            sudo service postgresql restart
            Stopping postgresql service:                               [  OK  ]
            Starting postgresql service:                               [  OK  ]
            
            sudo service ambari-server restart
            Using pytho
            Kafka Consumer-java API running on local machine
            Lines of Code : 63dot img10License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            package com.hortonworks.example.kafka.consumer;
            import org.apache.kafka.clients.CommonClientConfigs;
            import org.apache.kafka.clients.consumer.ConsumerConfig;
            import org.apache.kafka.clients.consumer.ConsumerRebalanceListener;
            import org.ap

            Community Discussions

            QUESTION

            Hortonworks Hadoop NN and RM heap stuck overloaded at high utilization, but no applications running? (java.io.IOException: No space left on device)
            Asked 2021-Apr-30 at 04:07

            Had some recent Spark jobs initiated from a Hadoop (HDP-3.1.0.0) client node that raised some

            Exception in thread "main" org.apache.hadoop.fs.FSError: java.io.IOException: No space left on device

            errors and now I see that the NN and RM heap appear stuck at high utilization levels (eg. 80-95%) despite there being to jobs pending or running in the RM/YARN UI.

            On the Ambari dashboard I see

            Yet in the RM UI, there appears to be nothing running:

            The errors that I see reported in most recent Spark jobs that failed are...

            ...

            ANSWER

            Answered 2021-Apr-30 at 04:07

            Running df -h and du -h -d1 /some/paths/of/interest on the machine doing the Spark calls just taking a guess from the "writing to local FS" and "No space on disk" messages in the errors (running clush -ab df -h / across all the hadoop nodes, I could see that the client node initiating the Spark jobs was the only one with high disk utilization), I found that there was only 1GB of disk space remaining on the machine that was calling the Spark jobs (due to other issues) that eventually threw this error for some of them and have since fixed that issue, but not sure if that is related or not (as my understanding is that Spark does the actual processing on other nodes in the cluster).

            I suspect that this was the problem, but if anyone with more experience could explain more what is going wrong under the surface here, that would be very helpful for future debugging and a better actual answer to this post. Eg.

            1. Why would the lack of free disk space on one of the cluster nodes (in this case, a client node) cause the RM heap to remain at such a high utilization percentage even when no jobs were reported running in the RM UI?
            2. Why would the lack of disk space on the local machine affect the Spark jobs (as my understanding is that Spark does the actual processing on other nodes in the cluster)?

            If the disk space on the local machine calling the spark jobs was indeed the problem, this question could potentially be marked as a duplicate to the question answered here: https://stackoverflow.com/a/18365738/8236733.

            Source https://stackoverflow.com/questions/66182012

            QUESTION

            How to remove the very large files under /hadoop/hdfs/journal/hdfsha/current/
            Asked 2021-Apr-21 at 14:26

            in our HDP cluster - version 2.6.5 , with ambari platform

            we noticed that /hadoop/hdfs/journal/hdfsha/current/ folder include huge files and more then 1000 files as

            ...

            ANSWER

            Answered 2021-Jan-20 at 07:36

            To clear out the space consumed by jornal edit, you are on right track. However the values are too less and if something goes wrong, you might loose data.

            The default value for dfs.namenode.num.extra.edits.retained and dfs.namenode.max.extra.edits.segments.retained is set to 1000000 and 10000 respectively.

            I would suggest following values:-

            Source https://stackoverflow.com/questions/65804491

            QUESTION

            Ambari server installation on centos7.4 get repos error
            Asked 2021-Mar-10 at 13:42

            I'm trying to get my own cluster with Ambari following this official guide https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide#space-menu-link-content. But I cannot copy repository on this command wget -O /etc/yum.repos.d/ambari.repo http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.5.1.0/ambari.repo I am using centos7.4

            As a result I have problem 403:Forbidden. I also tried to get this page via browser but I got an error.

            I also tried to use one of this repos https://docs.cloudera.com/HDPDocuments/Ambari-2.2.0.0/bk_Installing_HDP_AMB/content/_ambari_repositories.html or similar but there's still nothing. Could you help me guys?)

            ...

            ANSWER

            Answered 2021-Mar-10 at 13:42

            "Effective Jan 31, 2021, all Cloudera software requires a subscription and must be accessed via the paywall", please check https://www.cloudera.com/downloads.html

            Source https://stackoverflow.com/questions/66088341

            QUESTION

            Is there a way to fix Ambari Web 2.7.1.0.0 build issue?
            Asked 2021-Mar-06 at 05:02
            Issue


            I am currently setting an Ambari cluster and getting stuck at the ambari-web build stage

            Environment ...

            ANSWER

            Answered 2021-Mar-06 at 05:02

            I'm sure you won't be able to do that. Because ambari will need to download the repo from hortonwork. Eventually it will still fail, since there is no public for the hortonwork from 31/01/2021.

            Source https://stackoverflow.com/questions/66396266

            QUESTION

            postgresql + how to print all hosts
            Asked 2021-Feb-07 at 08:05

            we have ambari cluster with postgresql

            we try to print all hosts in the cluster as the following , but seems my select command not right

            ...

            ANSWER

            Answered 2021-Feb-07 at 08:05

            the command is ( with details of mem )

            Source https://stackoverflow.com/questions/66085719

            QUESTION

            Apache Knox Ldap Integration failed
            Asked 2021-Feb-01 at 05:15

            I am using Apacheknox version 1.0.0. I have tried to authenticate knox uiusing Ldap user. I have tried with following changes in knox

            ...

            ANSWER

            Answered 2021-Jan-31 at 17:59

            LDAP: error code 49 - INVALID_CREDENTIALS Means, three things, Username/password is incorrect or the account is locked. You are having this error for Bind user.

            You need to verify you systemUsername and systemPassword in configured topology.

            A tool ldapsearch can be useful to verify credentials for Bind user.

            main.ldapRealm.userDnTemplate should be like following

            Source https://stackoverflow.com/questions/65949164

            QUESTION

            hadoop cluster with active standby namenode + gap in the edit log
            Asked 2021-Jan-24 at 11:07

            we have ambari cluster , HDP version 2.6.5

            cluster include management of two name-node ( one is active and the secondary is standby )

            and 65 datanode machines

            we have problem with the standby name-node that not started and from the namenode logs we can see the following

            ...

            ANSWER

            Answered 2021-Jan-21 at 02:01

            There are many causes for this, However, check this article this should help.

            Follow exact steps in exact orders mentioned in article.

            In short the error means namenode matadata is damaged/corrupted.

            Source https://stackoverflow.com/questions/65796188

            QUESTION

            Apache Ambari not starting in hortonworks sandbox - probably 8080 is locked
            Asked 2021-Jan-07 at 15:45

            I have HDP from hortonworks sandbox in a virtual box. My host OS is mac10.15.7. I am getting the hortonworks Sandbox home page at localhost:1080. However, when i try to go Ambari UI at localhost:8080 it throws error 404 as shown below

            I just check to see if port 8080 is locked by some process and I get as below:

            ...

            ANSWER

            Answered 2021-Jan-07 at 13:05

            @Jio

            I ran into similar before, but its been too long and I cannot locate my notes. From what I recall, you need to use ip/hostname not localhost. You will find info that suggest the ambari url may look like:

            Source https://stackoverflow.com/questions/65604001

            QUESTION

            Can I apply Ambari on a Running cluster?
            Asked 2020-Dec-22 at 08:29

            I was looking for how to monitor Hadoop clusters more conveniently, and then I came across something called Ambari.
            I want to apply Apache Ambari to my running Hadoop cluster. Is it possible to apply Apache Ambari to a running Hadoop cluster? If this is not possible, are there any future patches planned?

            ...

            ANSWER

            Answered 2020-Nov-12 at 12:54

            @Coldbrew No. Ambari should be installed on a fresh cluster. If you indeed need to use ambari hadoop, I would recommend make a new ambari cluster w/ hadoop configured as close to possible as your existing hadoop and then migrating the native hadoop data to the new platform.

            Source https://stackoverflow.com/questions/64798762

            QUESTION

            hue installation - make apps - failed
            Asked 2020-Dec-16 at 02:11

            I am following this guide for installing HUE on HDP 3.1.4 , centos 7

            https://gethue.com/configure-ambari-hdp-with-hue/

            at building : sudo make apps ,it fails with :

            ...

            ANSWER

            Answered 2020-Dec-16 at 02:11

            in installed the latest versions from https://nodejs.org/en/download/ and it is working:

            centos 7:

            Source https://stackoverflow.com/questions/63149107

            Community Discussions, Code Snippets contain sources that include Stack Exchange Network

            Vulnerabilities

            CVE-2017-5642 CRITICAL
            During installation of Ambari 2.4.0 through 2.4.2, Ambari Server artifacts are not created with proper ACLs.
            Apache Ambari, version 2.5.0 to 2.6.2, passwords for Hadoop credential stores are exposed in Ambari Agent informational log messages when the credential store feature is enabled for eligible services. For example, Hive and Oozie.
            Apache Ambari, versions 1.4.0 to 2.6.1, is susceptible to a directory traversal attack allowing an unauthenticated user to craft an HTTP request which provides read-only access to any file on the filesystem of the host the Ambari Server runs on that is accessible by the user the Ambari Server is running as. Direct network access to the Ambari Server is required to issue this request, and those Ambari Servers that are protected behind a firewall, or in a restricted network zone are at less risk of being affected by this issue.
            CVE-2014-3582 CRITICAL
            In Ambari 1.2.0 through 2.2.2, it may be possible to execute arbitrary system commands on the Ambari Server host while generating SSL certificates for hosts in an Ambari cluster.
            In Ambari 2.4.x (before 2.4.3) and Ambari 2.5.0, an authorized user of the Ambari Hive View may be able to gain unauthorized read access to files on the host where the Ambari server executes.
            In Ambari 2.2.2 through 2.4.2 and Ambari 2.5.0, sensitive data may be stored on disk in temporary files on the Ambari Server host. The temporary files are readable by any user authenticated on the host.
            CVE-2016-6807 CRITICAL
            Custom commands may be executed on Ambari Agent (2.4.x, before 2.4.2) hosts without authorization, leading to unauthorized access to operations that may affect the underlying system. Such operations are invoked by the Ambari Agent process on Ambari Agent hosts, as the user executing the Ambari Agent process.
            Apache Ambari 2.x before 2.4.0 includes KDC administrator passwords on the kadmin command line, which allows local users to obtain sensitive information via a process listing.
            Apache Ambari before 2.1, as used in IBM Infosphere BigInsights 4.x before 4.1, stores a cleartext BigSheets password in a configuration file, which allows local users to obtain sensitive information by reading this file.
            Apache Ambari before 2.1, as used in IBM Infosphere BigInsights 4.x before 4.1, includes cleartext passwords on a Configs screen, which allows physically proximate attackers to obtain sensitive information by reading password fields.
            The agent in Apache Ambari before 2.1.2 uses weak permissions for the (1) /var/lib/ambari-agent/data and (2) /var/lib/ambari-agent/keys directories, which allows local users to obtain sensitive information by reading files in the directories.
            Open redirect vulnerability in Apache Ambari before 2.1.2 allows remote attackers to redirect users to arbitrary web sites and conduct phishing attacks via a URL in the targetURI parameter.
            Apache Ambari before 2.0.2 or 2.1.x before 2.1.1 allows remote authenticated users to gain administrative privileges via unspecified vectors, possibly related to changing passwords.
            Cross-site scripting (XSS) vulnerability in Apache Ambari before 2.1.0 allows remote authenticated cluster operator users to inject arbitrary web script or HTML via the note field in a configuration change.

            Install ambari

            You can download it from GitHub, Maven.
            You can use ambari like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the ambari component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

            Support

            For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries
            Install
            Maven
            Gradle
            CLONE
          • HTTPS

            https://github.com/apache/ambari.git

          • CLI

            gh repo clone apache/ambari

          • sshUrl

            git@github.com:apache/ambari.git

          • Stay Updated

            Subscribe to our newsletter for trending solutions and developer bootcamps

            Agree to Sign up and Terms & Conditions

            Share this Page

            share link