gradient-checkpointing | Make huge neural nets fit in memory

by cybertronai Python Version: Current License: MIT

X-Ray Key Features Code Snippets(3)Community Discussions Vulnerabilities Install Support

kandi X-RAY | gradient-checkpointing Summary

gradient-checkpointing is a Python library. gradient-checkpointing has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has medium support. You can download it from GitHub.

Status: Maintenance (expect bug fixes and minor updates).

Support

Quality

Security

License

Reuse

Support

gradient-checkpointing has a medium active ecosystem.

It has 2440 star(s) with 261 fork(s). There are 82 watchers for this library.

It had no major release in the last 6 months.

There are 27 open issues and 14 have been closed. On average issues are closed in 46 days. There are 1 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of gradient-checkpointing is current.

Quality

gradient-checkpointing has 0 bugs and 0 code smells.

Security

gradient-checkpointing has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

gradient-checkpointing code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

gradient-checkpointing is licensed under the MIT License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

gradient-checkpointing releases are not available. You will need to build from source code and install.

Build file is available. You can build the component from source.

Installation instructions, examples and code snippets are available.

gradient-checkpointing saves you 1698 person hours of effort in developing the same functionality from scratch.

It has 3763 lines of code, 285 functions and 25 files.

It has high code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed gradient-checkpointing and discovered the below as its top functions. This is intended to give you an instant insight into gradient-checkpointing implemented functionality, and help decide if they suit your requirements.

Compute the gradient of the gradients
Compute the topos sorted list of tensors
Creates a generator of all the ops in the current scope
Print debug information
Format a list of ops
Wrapper for tf gradients
Returns True if o is an iterable
Convert an iterable to a list of operations
Convert a tensor_op to an op
Returns a list of backward walk operations
Add control inputs to wait_to_loop

Get all kandi verified functions for this library.

gradient-checkpointing Key Features

No Key Features are available at this moment for gradient-checkpointing.

gradient-checkpointing Examples and Code Snippets

Training GPT-2 on a Russian language corpus,2. Training environment

Python

Lines of Code : 12

License : No License

Copy

import horovod.tensorflow as hvd
...
hvd.init()
...
config.gpu_options.visible_device_list = str(hvd.local_rank())
...
bcast = hvd.broadcast_global_variables(0).
...
opt = tf.train.AdamOptimizer(decayed_lr)
opt = hvd.DistributedOptimizer(opt)

horovo

Citation:

Python

Lines of Code : 6

License : Permissive (MIT)

Copy

@inproceedings{fenghuang2021,
  title={Optimal Gradient Checkpoint Search for Arbitrary Computation Graphs},
  author={Jianwei Feng and Dong Huang},
  booktitle={CVPR},
  year={2021}
}

Gradient checkpointing

Python

Lines of Code : 6

License : No License

Copy

def call(self, x, past):
    @gradient_checkpointing.recompute_grad
    def inner(x):
        # ops go here
        return y
    return inner(x)

Community Discussions

No Community Discussions are available at this moment for gradient-checkpointing.Refer to stack overflow page for discussions.

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install gradient-checkpointing

Also, when running the tests, make sure that the CUDA Profiling Tools Interface (CUPTI) can be found, e.g. by running export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/local/cuda/extras/CUPTI/lib64".

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: