webhook | lightweight incoming webhook server to run shell commands | DevOps library

I'm pulling my hairs here. I have a Google Assistant application that I build with Jovo 4 and Google Actions Builder.

The goal is to create a HelpScene, which shows some options that explain the possibilities/features of the app on selection. This is the response I return from my Webhook. (This is Jovo code, but doesn't matter as this returns a JSON when the Assistant calls the webhook.)

I have a public Github repo and an on-prem Jenkins server (this could not be changed)

I want a Jenkins job to start every time a PR is opened in the github repo.

Now, if my Jenkins had public access I will simply set up a webhook on github, but since github can not access my Jenkins what can I do to trigger the job?

Is there an option to set a webhook the other way around? meaning the Jenkins will check every few seconds/minutes if a PR was opened and if so it will trigger the job.

Im trying to make an ingress for the minikube dashboard using the embedded dashboard internal service.

I enabled both ingress and dashboard minikube addons.

I also wrote this ingress YAML file :

I am developing a C++ application, where the program run endlessly, allocating and freeing millions of strings (char*) over time. And RAM usage is a serious consideration in the program. This results in RAM usage getting higher and higher over time. I think the problem is heap fragmentation. And I really need to find a solution.

You can see in the image, after millions of allocation and freeing in the program, the usage is just increasing. And the way I am testing it, I know for a fact that the data it stores is not increasing. I can guess that you will ask, "How are you sure of that?", "How are you sure it's not just a memory leak?", Well.

This test run much longer. I run malloc_trim(0), whenever possible in my program. And it seems, application can finally return the unused memory to the OS, and it goes almost to zero (the actual data size my program has currently). This implies the problem is not a memory leak. But I can't rely on this behavior, the allocation and freeing pattern of my program is random, what if it never releases the memory ?

• I said memory pools are a bad idea for this project in the title. Of course I don't have absolute knowledge. But the strings I am allocating can be anything between 30-4000 bytes. Which makes many optimizations and clever ideas much harder. Memory pools are one of them.
• I am using GCC 11 / G++ 11 as a compiler. If some old versions have bad allocators. I shouldn't have that problem.
• How am I getting memory usage ? Python psutil module. proc.memory_full_info()[0], which gives me RSS.
• Of course, you don't know the details of my program. It is still a valid question, if this is indeed because of heap fragmentation. Well what I can say is, I am keeping a up to date information about how many allocations and frees took place. And I know the element counts of every container in my program. But if you still have some ideas about the causes of the problem, I am open to suggestions.
• I can't just allocate, say 4096 bytes for all the strings so it would become easier to optimize. That's the opposite I am trying to do.

So my question is, what do programmers do(what should I do), in an application where millions of alloc's and free's take place over time, and they are of different sizes so memory pools are hard to use efficiently. I can't change what the program does, I can only change implementation details.

Bounty Edit: When trying to utilize memory pools, isn't it possible to make multiple of them, to the extent that there is a pool for every possible byte count ? For example my strings can be something in between 30-4000 bytes. So couldn't somebody make 4000 - 30 + 1, 3971 memory pools, for each and every possible allocation size of the program. Isn't this applicable ? All pools could start small (no not lose much memory), then enlarge, in a balance between performance and memory. I am not trying to make a use of memory pool's ability to reserve big spaces beforehand. I am just trying to effectively reuse freed space, because of frequent alloc's and free's.

Last edit: It turns out that, the memory growth appearing in the graphs, was actually from a http request queue in my program. I failed to see that hundreds of thousands of tests that I did, bloated this queue (something like webhook). And the reasonable explanation of figure 2 is, I finally get DDOS banned from the server (or can't open a connection anymore for some reason), the queue emptied, and the RAM issue resolved. So anyone reading this question later in the future, consider every possibility. It would have never crossed my mind that it was something like this. Not a memory leak, but an implementation detail. Still I think @Hajo Kirchhoff deserves the bounty, his answer was really enlightening.

I want to set up a telegram webhook and I need to receive https requests. How can I make my ip an accessible url?

I was doing this in Python with aiohttp.

In GitHub Enterprise, we have Project A under Organization A. When I submit a PR (pull request) to Project A, the continuous-integration/jenkins/pr-merge is triggered which runs a Jenkins pipeline to build the code and perform unit tests. This allows us to prevent the PR from being merged into master if the unit tests fail.

For example, this is what I see on a PR for Project A in GitHub that includes a broken unit test:

Now I am trying to configure Project B under Organization B to behave the same way. However, it is not working. This is what I see on a PR for Project B in GitHub that includes a broken unit test:

Notice that Project B's PR did not kick off the continuous-integration/jenkins/pr-merge.

GitHub -> Settings -> Branches -> Branch protection rules

Project A in GitHub has a branch protection rule for master with only one setting enabled:

• Require pull request reviews before merging

Interestingly, the "Require status checks to pass before merging" setting is not enabled. Out of curiosity, I enabled it (without saving it) and noticed that "continuous-integration/jenkins/pr-merge" showed up below it as an option.

I configured Project B to have the exact same branch protection rule for master with only "Require pull request reviews before merging" enabled. Out of curiosity, I enabled "Require status checks to pass before merging" (without saving it) and it doesn't even show continuous-integration/jenkins/pr-merge as an option. It just says "No status checks found. Sorry, we couldn’t find any status checks in the last week for this repository."

GitHub -> Settings -> Hooks -> Webhooks

Project A in GitHub has a webhook configured with:

• Payload URL https://jenkins.mycompany.com/github-webhook/
• Content type application/json
• Let me select individual events: Pull requests, Pushes, Repositories are checked
• Active: checked

I created a webhook for Project B with the exact same settings. After I submitted a PR for Project B, I see a couple of items under "Recent Deliveries" for Project B's webhook with green checkmarks and "200" response codes, so I think it is configured correctly.

CloudBees Jenkins Enterprise

In Jenkins Enterprise, Project A's pipeline is of type "GitHub Organization" and has the following settings:

• API endpoint: kubernetes-cbs-automation (https://git.mycompany.com/api/v3)
• Credentials: [credentials specific to Project A]
• Owner: [Project A's GitHub organization]
• Behaviors: Repositories: Filter by name (with regular expression): Regular expression: [name of Project A's GitHub repo]
• Behaviors: Within repository: Discover pull requests from origin: Strategy: Merging the pull request with the current target branch revision
• Project Recognizers: Pipeline Jenkinsfile: Script Path: ci-cd/jenkins/ProjectA-pipeline.groovy
• Property strategy: All branches get the same properties
• Scan Organization Triggers: "Periodically if not otherwise run" checked: Interval: 1 day
• Orphaned Item Strategy: "Discard old items" checked
• Child Orphaned Item Strategy: Strategy: Inherited
• Child Scan Triggers: "Periodically if not otherwise run" checked: Interval: 1 day
• Automatic branch project triggering: Branch names to build automatically: .*

I created an item under Project B in Jenkins Enterprise of type "GitHub Organization" with the same settings (except any settings specific to Project A were replaced with the appropriate Project B specific settings).

Given that GitHub PRs for Project B are failing to launch the continuous-integration/jenkins/pr-merge, it seems like there is some configuration that I am missing. Unfortunately, our GitHub/Jenkins admins have not been able to figure out what is wrong.

UPDATE

We have confirmed that Project B is actually launching a build on the Jenkins agent when a PR is submitted. The problem is that GitHub is not showing the continuous-integration/jenkins/pr-merge on the web page for the PR. We need that so the PR can be blocked if the build fails, and also so that we can quickly see what went wrong.

I've been working on a project which so far has just involved building some cloud infrastructure, and now I'm trying to add a CLI to simplify running some AWS Lambdas. Unfortunately both the sdist and wheel packages built using poetry build don't seem to include the dependencies, so I have to manually pip install all of them to run the command. Basically I

1. run poetry build in the project,
2. cd "$(mktemp --directory)",
3. python -m venv .venv,
4. . .venv/bin/activate,
5. pip install /path/to/result/of/poetry/build/above, and then
6. run the new .venv/bin/ executable.

At this point the executable fails, because pip did not install any of the package dependencies. If I pip show PACKAGE the Requires line is empty.

The Poetry manual doesn't seem to specify how to link dependencies to the built package, so what do I have to do instead?

I am using some optional dependencies, could that be interfering with the build process? To be clear, even non-optional dependencies do not show up in the package dependencies.

pyproject.toml:

Hey guys so I'm building a websocket using flask and I'm currently creating a new thread for every request so that thread can do some heavy processing and not take too much time to return something from the request. The problem is that at some point i get so many threads open that it starts causing problems and i was wondering if i could setup some queue in flask to limit the app for creating like 8 thread each time only.

My code:

I'm working on connecting Cloud Error Reporting to an on request Cloud Function (to eventually liaise with the JIRA API).

I have been able to successfully receive Monitoring Notification Channels' test notification to the webhook, however events triggering the email notification channel do not also trigger the webhook.

Is there a way to dispatch on error reporting event to a web URL or pubsub job with error_group attached?