hms-mirror | Copy Hive tables definitions to Compute Cluster

by dstreev Java Version: 1.5.1.7-SNAPSHOT License: Apache-2.0

X-Ray Key Features Code Snippets Community Discussions Vulnerabilities Install Support

kandi X-RAY | hms-mirror Summary

hms-mirror is a Java library. hms-mirror has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. You can download it from GitHub.

"hms-mirror" is a utility used to bridge the gap between two clusters and migrate hive metadata. HMS-Mirror is distributed under the AGPLv3 license. The application will migrate hive metastore data (metadata) between two clusters. With SQL and EXPORT_IMPORT data strategies, we can move data between the two clusters. While this process functions on smaller datasets, it isn't too efficient for larger datasets. For the default strategy SCHEMA_ONLY, we can migrate the schemas and sync metastore databases, but the DATA movement is NOT a function of this application. The application does provide a workbook for each database with SOURCE and TARGET locations. The output reports are written in Markdown. If you have a client Markdown Renderer like Marked2 for the Mac or Typora which is cross-platform, you'll find a LOT of details in the output reports about what happened. If you can't install a render, try some web versions Stackedit.io. Copy/paste the contents to the report md files.

Support

Quality

Security

License

Reuse

Support

hms-mirror has a low active ecosystem.

It has 11 star(s) with 5 fork(s). There are 5 watchers for this library.

It had no major release in the last 12 months.

There are 14 open issues and 40 have been closed. On average issues are closed in 32 days. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of hms-mirror is 1.5.1.7-SNAPSHOT

Quality

hms-mirror has 0 bugs and 0 code smells.

Security

hms-mirror has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

hms-mirror code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

hms-mirror is licensed under the Apache-2.0 License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

hms-mirror releases are available to install and integrate.

Build file is available. You can build the component from source.

Installation instructions, examples and code snippets are available.

It has 5455 lines of code, 306 functions and 37 files.

It has high code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed hms-mirror and discovered the below as its top functions. This is intended to give you an instant insight into hms-mirror implemented functionality, and help decide if they suit your requirements.

Performs migration
Build table schema
Builds the intermediate definition
Do the intermediate transfer
Get the tables for this table
Get the table for a database
Gets the cluster with the given environment
Gets a database connection
Create databases
Build SQL statements for Hive
Build COMMON SQL
Checks if ocfs are equal
Returns the database for the given database
Checks if a database is transolidate
Get the translation database location
Deletes the retry state
Returns a hash code for the current node
Retrieves the location of a translation table from a database
Runs the wizard
The main test method
Prefix a table with a given prefix
Fetch the report template
Returns the SQL script to execute
Gets the serde path for a table
Checks whether a table definition is migrated
Translate a table

Get all kandi verified functions for this library.

hms-mirror Key Features

No Key Features are available at this moment for hms-mirror.

hms-mirror Examples and Code Snippets

No Code Snippets are available at this moment for hms-mirror.

Community Discussions

No Community Discussions are available at this moment for hms-mirror.Refer to stack overflow page for discussions.

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install hms-mirror

If you install both options, your environment PATH will determine which one is run. Make note of this because an upgrade may not be reachable.
Remove previous install directory rm -rf hms-mirror-install
Expand the tarball tar zxvf hms-mirror-dist.tar.gz. This produces a child hms-mirror-install directory.
Two options for installation: As the root user (or sudo), run hms-mirror-install/setup.sh. This will install the hms-mirror packages in /usr/local/hms-mirror and create symlinks for the executables in /usr/local/bin. At this point, hms-mirror should be available to all user and in the default path. As the local user, run hms-mirror-install/setup.sh. This will install the hms-mirror packages in $HOME/.hms-mirror and create symlink in $HOME/bin. Ensure $HOME/bin is in the users path and run hms-mirror.
hms-mirror requires a configuration file describing the LEFT (source) and RIGHT (target) cluster connections. There are two ways to create the config:.
hms-mirror --setup - Prompts a series of questions about the LEFT and RIGHT clusters to build the default configuration file.
Use the default config template as a starting point. Edit and place a copy here $HOME/.hms-mirror/cfg/default.yaml.

Support

If each JDBC endpoint is Kerberized and the connection to the LEFT or RIGHT is successful, both NOT both, and the program seems to hang with no exception... it's most likely that the Kerberos ticket isn't TRUSTED across the two environments. You will only be able to support a Kerberos connection to the cluster where the ticket is trusted. The other cluster connection will need to be anything BUT Kerberos. Add --show-cp to the hms-mirror command line to see the classpath used to run. The argument --hadoop-classpath allows us to replace the embedded Hadoop Libs (v3.1) with the libs of the current platform via a call to hadoop classpath. This is necessary to connect to kerberized Hadoop v2/Hive v1 environments.

Find more information at: