FFMpegSharp | great way to use FFMpeg | Video Utils library

Given that you need this to work cross platform, as @Andy said the best solution is ffmpeg. You have two choices:

1. Invoke the ffmpeg command line utility from your web process; or
2. Compile the libav library suite (which underlies ffmpeg) to each native platform your code might be run on - Windows, Linux, etc. - make a native DLL wrapper, and use P/Invoke in your ASP.NET project to access it.

Command Line Utility

The command line utility is very easy to use and well documented. Documentation is here. Your basic approach would be to include ffmpeg.exe in your web project (make sure you have a version for each platform), and use Process.Start to invoke it, using the command line arguments to point it to your video file and configure the output. Once the output is finished, you can serve it by returning with File like in your example. There are also some open source .NET wrappers like this one that could save you some of the work. Unfortunately the command line utility doesn't offer much (any) control once started, or a programmatic way of determining progress. However, these issues should not be a problem if you follow my recommendation at the end.

Libav

If you do need or want total control, however - including frame by frame transcoding, progress reporting, etc. - then you would need to use libav. Before going further, note that you need to use at least some C/C++ to use libav. That means your server code is going to have to run with full trust, and you WILL be susceptible to the security risks of running native code. (Though the same would be true if you used ffmpeg.exe, but at least in that case you don't run the risk of introducing NEW security risks through your own code).

Also know that you can't just find nice clean, always up-to-date downloadable binaries for every platform (at least one reason for which is fear of patent lawsuits). Instead you have to build it yourself for every platform your code might run on. Find your platform(s) on here and then follow the instructions to the letter. If you make a single deviation no matter how small, you won't be able to build it, and you will pull your hair out figuring out why.

Once you have the builds, then your next major task is to expose the APIs you need to your C# code. The documentation for the libav APIs is not a model of clarity. They more or less assume you will look at the code for the ffmpeg command line utility to figure out how to use libav, and that's what you should do. But if you invest the time (days if not weeks) to construct the builds and learn the APIs, you will become a Master of Media. There is virtually nothing imaginable that you can't do using libav.

Now you're finally ready to integrate this into your app. Again you can take two approaches. If you're quite comfortable with C/C++, you should make a new C/C++ DLL project, link the libav DLLs to it, and do most of the heavy lifting there and just export couple of entrypoint functions that you can invoke from C#. Alternatively, you can P/Invoke directly to the libav DLLs, but you will need to do a ton of scaffolding of data structures and functions in C#. Unless you're extremely comfortable with marshalling, I would not attempt this.

In fact, I'm going to recommend going the command line utility route, because -

You Shouldn't Try to Transcode On The Fly Anyway

With all that out of the way, let's talk about your actual gameplan. You said you need to "dynamically" convert the video based on what the client wants/can receive. No one does this. What they do do is create multiple versions of the videos in advance and save each one on the server e.g., a 1080p, 720p, 480p, and maybe even 240p version. Then, the client application monitors the connection quality and, also considering the user's preference, directs the media player to the desired version. The server always serves the version requested and doesn't convert on the fly. Unless you're talking about streaming live events - and if so then that's beyond the scope of my expertise - this is what you should do.

So, what I would advise is use the ffmpeg utility to create different versions of the videos in advance - as part of an import or upload process for example. Track the videos in a database including what versions are available for each. Give the client a way to obtain this information. Then when it comes time to serve the videos, you just serve the one the client requests in a query parameter. Put the logic for determining the desired version on the client - either connection speed and/or user preference.

And Don't Forget to Support Content-Range Requests

Finally, you probably don't want to just use File to serve the media unless the users are just going to download the files for offline viewing. Assuming people are going to play videos in a browser, you need your API to accept content-range request headers. Here's a pretty good example of how to do that. Provided you implement it correctly, web browser media players will be able to play, seek, etc., transparently. If for whatever reason the format needs to change, just redirect the URL of the media player to the appropriate version, keep the position the same, and resume playing, and the user will barely notice a skip.

Hope at least some of this helps!