cave_miner | Search for code cave in all binaries | Reverse Engineering library

OK, I thought this was a nice thing to work on. I must say that I miss the context, I have no idea what this will be used for. Context is important for understanding the code. I assume it is some kind of game?

The first thing to do is to get the array definitions out of the way:

^{See also: This follow-up question.}

That your cast's operand is a COM object (as evidenced by System.__ComObject being reported as the object type in the error message) may be the source of the problem, because I don't think PowerShell can cast COM objects to other types.

However, given that PowerShell can dynamically discover members on objects, in many cases where C# requires casts, PowerShell doesn't (and casts to interfaces are no-ops in PowerShell, except when guiding method overload resolution). Similarly, there's no (strict) need to type variables.^[1]

Thus, as you've confirmed, simply omitting the cast of $thMainPipe.InnerObject to type [Microsoft.SqlServer.Dts.Pipeline.Wrapper.MainPipe] worked:

So the answer to this question was more or less explained in this article. Basically the issue was that the code was indeed obfuscated properly but there was still Kotlin Metadata and Android Studio was reconstructing the code based on this Metadata.

1. Pass cookies={"PHPSESSID": "3r7ql7poiparp92ia7ltv8nai5"} instead of headers={"cookie": "PHPSESSID=3r7ql7poiparp92ia7ltv8nai5"}.
   This is because the requests library does headers.pop('Cookie', None) upon redirect.
2. Retry if resp.url is not f"https://www.fadedpage.com/books/{bookID}/{fileType}.php".
   This is because the server first redirects link.php with a different bookID to showbook.php.
3. A download of downloadFile("20170817", "html") contains the text "The First Part of this book is intended for pupils", not "woodland slope behind St. Pierre-les-Bains" that is contained in a download of downloadFile("20130603", "html").

In a way, this is how Time-based one-time passwords (TOTPs) work, so you can use a similar solution.

To get a time value that changes every N seconds, you can use the following formula.

floor(timestamp / N)

Then, you can either turn that into a string or interpret it as bytes. Just pass it to something like HMAC in order to turn it into a pseudo-random value.

HMAC(SecretKey, floor(timestamp / N))

Here's a simple implementation in Python. This should be fairly similar in other languages too.

I think the getElementsByClassName may have problems with multiple classes. So I used querySelectorAll and it works.

Maybe it was a c+p error, but in your example code a ); was missing.

The purpose of it is to be able to send that key everytime I call a webservice that I've made, so I'm sure (or almost sure) that the call comes from the original app that I'm making and that will be published on the Play Store, and not from elsewhere.

This is a very hard task to achieve, but not impossible one and here is where one needs to make a deep dive in mobile API security and understand the mechanics behind it.

It's fundamental to have a clear understand between the difference of who is in the API request versus what is making that API request, otherwise any security solution you may devise/use may not have the intended results.

I wrote a series of articles around API and Mobile security, and in the article Why Does Your Mobile App Need An Api Key? you can read in detail the difference between who and what is accessing your API server, but I will extract here the main takes from it:

The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?

The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.

So, you need to think about the who as the user your API server will be able to Authenticate and Authorize access to the data, and you need to think about the what as the software making that request in behalf of the user.

I also don't want to store my key in my code, also because it could easily be retrieved via reverse engineering.

That's very true, it's more or less easily achieved depending on the method used to hide the API key, as per the ones you mention:

I've also checked the other methods explained here, but they look like the API key could quite easily be retrieved thanks to reverse engineering.

No matter how secure the API key has been stored, be it in the Android Keystore, encrypted, obfuscated, etc, at some point the API key will need to be in plain text to be sent on the API request header, and in this moment it will be vulnerable to be extracted via static reverse engineering, via a MitM attack or via an instrumentation framework

I have wrote the article How to Extract an API key from a Mobile App with Static Binary Analysis to illustrate how easy it can be done:

The range of open source tools available for reverse engineering is huge, and we really can't scratch the surface of this topic in this article, but instead we will focus in using the Mobile Security Framework(MobSF) to demonstrate how to reverse engineer the APK of our mobile app. MobSF is a collection of open source tools that present their results in an attractive dashboard, but the same tools used under the hood within MobSF and elsewhere can be used individually to achieve the same results.

During this article we will use the Android Hide Secrets research repository that is a dummy mobile app with API keys hidden using several different techniques.

I also wrote another article to achieve it during runtime, Steal that Api Key with a Man in the Middle Attack:

In order to help to demonstrate how to steal an API key, I have built and released in Github the Currency Converter Demo app for Android, which uses the same JNI/NDK technique we used in the earlier Android Hide Secrets app to hide the API key.

So, in this article you will learn how to setup and run a MitM attack to intercept https traffic in a mobile device under your control, so that you can steal the API key. Finally, you will see at a high level how MitM attacks can be mitigated.

An instrumentation framework can also be used during runtime to hook into the code that uses the API key in order to extract it. For example with the popular Frida framework:

Inject your own scripts into black box processes. Hook any function, spy on crypto APIs or trace private application code, no source code needed. Edit, hit save, and instantly see the results. All without compilation steps or program restarts.

So, no matter what it's done to secure the API key, once it's on the API request will be vulnerable to be extracted.

Anything that runs on the client side and needs some secret to access an API can be abused in different ways and you can learn more on this series of articles about Mobile API Security Techniques. This articles will teach you how API Keys, User Access Tokens, HMAC and TLS Pinning can be used to protect the API and how they can be bypassed.

I recommend you to read this answer I gave to the question How to secure an API REST for mobile app?, especially the sections Hardening and Shielding the Mobile App, Securing the API Server and A Possible Better Solution.

The possible best solution for your problem is known by Mobile App Attestation, that will let your backend know that what is making the request is indeed a genuine and untampered version of your mobile app, as you wish to achieve:

The purpose of it is to be able to send that key everytime I call a webservice that I've made, so I'm sure (or almost sure) that the call comes from the original app that I'm making and that will be published on the Play Store, and not from elsewhere.

In any response to a security question I always like to reference the excellent work from the OWASP foundation.

OWASP API Security Top 10

The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.

OWASP Mobile Security Project - Top 10 risks

The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.

OWASP - Mobile Security Testing Guide:

The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.

It is possible but I'm unaware of any tools that do this.

Protocol Buffers (protos) including gRPC service definitions are compiled by protoc into language-specific sources. You're looking for a decompiler.

We know that the process is invertible because it works; we're able to send messages using generated sources -- even across languages -- to peers.

Obfuscation is all about renaming your human-readable classes and functions into something meaningless to a human. Machines don't care about names but people trying to reverse engineer your code would have a much harder time.

On the other hand, when your app crashes, the Google Play Developer Console would log this crash for you to inspect and debug. But as the final user has an obfuscated version of your app, the report sent to you is written with meaningless names and you cannot debug it.

Now, the debug symbols map are used internally by the Play Console to resymbolize the crash report into human readable class names so you can debug it easily.

TLDR: Upload the debug symbols. They allow you (the developer) to debug ofuscated crash reports and are only available (indirectly) to you, not people trying to reverse engineer your app