64k intro in c#
category: code [glöplog]
Someone had developed 64k intro using .net framework ?
I don't think this is viable/possible at all. Please correct me if I'm wrong.
I think it would be tough; executable compressors do exist, but I'm not sure their compression ratio vs the .NET code size makes it viable.
I *think* it's possible (I have been thinking about this for quite a while), but it'll need some effort.
.NETZ seems to just compress the binary and add some hacks to make it work, like a basic dropper. This is far from ideal, as the file format is rather redundant and stupid (UTF-16 strings, really?).
.NET binaries consist of the bytecode (which could benefit from a transformation à la kkrunchy), and the metadata tables (lots of uninteresting stuff you probably want to get rid of, unless you're using reflection). Libraries such as dnlib can be used for this. (Mono.Cecil only performs 'high-level' bytecode modification, so it isn't suited for this.)
There are multiple ways to approach this:
* 'nondestructive transformation' of the binary: make the data more compressible while leaving it legible for the parser, then prepend a smaller CorExeMain stub.
* Store the data in a custom format, reconstruct a valid file at runtime, then execute it.
* This can be done using both a .NET dropper or a native dropper. Both have up- and downsides. (Cross-platform, code can be loaded through reflection, but it's large, vs. small, but harder to set up, and both the .NET Framework and Mono have to be taken into account.) (Though, I'm probably the only one here who cares about Mono.)
* Compile from source at runtime. Most .NET installations come with csc (command-line C# compiler) automatically, but I don't think this is the case for newer .NET versions, although I'm not sure (it's been ages since I last used Windows). Most Linux distros (and probably BSDs, too) ship the Mono runtime and the entire thing separately as well.
Of course, a compressor has to be fit somewhere in between, and getting a window + an OpenGL context set up probably won't be easy either, as XNA/MonoGame/OpenTK/... won't be available. (You probably want runtime code generation to create all the OpenGL functions. Either do this in the dropper, or using System.Reflection.Emit)
.NETZ seems to just compress the binary and add some hacks to make it work, like a basic dropper. This is far from ideal, as the file format is rather redundant and stupid (UTF-16 strings, really?).
.NET binaries consist of the bytecode (which could benefit from a transformation à la kkrunchy), and the metadata tables (lots of uninteresting stuff you probably want to get rid of, unless you're using reflection). Libraries such as dnlib can be used for this. (Mono.Cecil only performs 'high-level' bytecode modification, so it isn't suited for this.)
There are multiple ways to approach this:
* 'nondestructive transformation' of the binary: make the data more compressible while leaving it legible for the parser, then prepend a smaller CorExeMain stub.
* Store the data in a custom format, reconstruct a valid file at runtime, then execute it.
* This can be done using both a .NET dropper or a native dropper. Both have up- and downsides. (Cross-platform, code can be loaded through reflection, but it's large, vs. small, but harder to set up, and both the .NET Framework and Mono have to be taken into account.) (Though, I'm probably the only one here who cares about Mono.)
* Compile from source at runtime. Most .NET installations come with csc (command-line C# compiler) automatically, but I don't think this is the case for newer .NET versions, although I'm not sure (it's been ages since I last used Windows). Most Linux distros (and probably BSDs, too) ship the Mono runtime and the entire thing separately as well.
Of course, a compressor has to be fit somewhere in between, and getting a window + an OpenGL context set up probably won't be easy either, as XNA/MonoGame/OpenTK/... won't be available. (You probably want runtime code generation to create all the OpenGL functions. Either do this in the dropper, or using System.Reflection.Emit)
I have done a test with a c# launcher that decompress ilcode (intro part) and run it.
I can have an executable <64k with a 3d metaball scene using opengl ...
Compression/Decompression is done using gzip from .net framework.
Problem is to remove automatically unless classes and methods.
That a start.
I can have an executable <64k with a 3d metaball scene using opengl ...
Compression/Decompression is done using gzip from .net framework.
Problem is to remove automatically unless classes and methods.
That a start.
you could try to use Unity's il2cpp tool which is how they are able to ship unity games on iOS, Switch, etc...
Here's a 4 year old blog post
il2cpp
Whether or not you can get close to 64k I have no idea. I'd try just "Hello World" to see if it generates something small or not
Here's a 4 year old blog post
il2cpp
Whether or not you can get close to 64k I have no idea. I'd try just "Hello World" to see if it generates something small or not
Or let's have more 128k or 256kb intro compos at parties.
Would be interesting to see battles between C# (or other language frameworks) and native c++ intros.
And larger filesize would possibly allow more interesting content (more sound samples, compressed meshes, etc.)
Would be interesting to see battles between C# (or other language frameworks) and native c++ intros.
And larger filesize would possibly allow more interesting content (more sound samples, compressed meshes, etc.)
Good thread. C# is my favourite programming language.
Bad post. C# is not my favourite programming language.
fxgen: hmm, that's interesting. I remember writing a tool that removes unreferenced metadata table entries from an assembly (in F#, because reasons), but idk where it is, and how well it worked. Though I suppose it isn't too difficult to roll your own, by using something like dnlib (which is more low-level than mono.cecil), as it's basically just graph traversal. you can then also change/zero out the names of the symbols (while leaving the references themselves in place properly), clear as many flags as possible to decrease entropy, etc etc.
I'm a bit too lazy to start hacking on a PoC, but you can always hit me up on IRC :p
greggman: that'd be cheating :p (make the same binary work on mono as well, I dare you.) plus, iirc, you need the paid version for this, which, double ew.
I'm a bit too lazy to start hacking on a PoC, but you can always hit me up on IRC :p
greggman: that'd be cheating :p (make the same binary work on mono as well, I dare you.) plus, iirc, you need the paid version for this, which, double ew.
Quote:
Compile from source at runtime. Most .NET installations come with csc (command-line C# compiler) automatically, but I don't think this is the case for newer .NET versions, although I'm not sure
FWIW, the Windows 10 installations I have flying around here all ship with
Code:
%windir%\Microsoft.NET\Framework64\v4.0.30319\csc.exe
I don't know how useful this is though. The compiler version itself is 4.8.3752.0 (on both 1903 and 1909), but it says it can only do C# up to version 5.
when using "System.Numerics" for vectors and matrices stuffs I have an 35 ko executable size ! I have my own class for maths before...
Porocyon: how to use dnlib to find unless methods ? Thanks.
Porocyon: how to use dnlib to find unless methods ? Thanks.
fxgen: basically,
1. find the entrypoint
2. in the entrypoint: for every local variable type, and for every instruction that uses a function/constructor call, property getter/setter, field, ..., add that function/ctor/field/getter/setter to a list, and add returntype/argtypes/... + the type that contains the fn/ctor/... to that list too (you can also use one list for fns, one for types, etc etc)
3. do n° 2 for every fn/ctor/getter/setter that was added to the list
4. GOTO 3 while you keep finding new fns/ctors/...
5. for every type/fn/ctor/... in the assembly: if it is NOT in the list, remove it from the assembly
if you want to see code that uses dnlib: a long time ago I used dnlib to inject code into Terraria, for a modloader, linky. 90% of it is MSIL patching and not modifying types, but here is a simple 'example', makes everything public.
you can also do these optimizations:
* inline functions that are used once or twice (no entry in the metadata table + same bytecode -> good compression)
* replace pinvoke calls by the 'calli' opcode
* replace memcpy/memset/... by cpblk/initblk
* replace property getter/setters by functions
* merge classes with only static fields/fns/... into 1
* remove useless attributes
* remove/change names of types/fns/... for better compression
* simplify MD tables: make everyting public, no 'static this'/'extension' methods, avoid generics & inheritance, ...
* use small ldi, br, ... opcodes (like br.s) when possible (dnlib does this for you: linky)
also, for debugging: use *dnSpy* for inspecting .exe/.dlls
again, meet me on IRC :p (#revision on ircnet, #titandemo on efnet, ...)
1. find the entrypoint
2. in the entrypoint: for every local variable type, and for every instruction that uses a function/constructor call, property getter/setter, field, ..., add that function/ctor/field/getter/setter to a list, and add returntype/argtypes/... + the type that contains the fn/ctor/... to that list too (you can also use one list for fns, one for types, etc etc)
3. do n° 2 for every fn/ctor/getter/setter that was added to the list
4. GOTO 3 while you keep finding new fns/ctors/...
5. for every type/fn/ctor/... in the assembly: if it is NOT in the list, remove it from the assembly
if you want to see code that uses dnlib: a long time ago I used dnlib to inject code into Terraria, for a modloader, linky. 90% of it is MSIL patching and not modifying types, but here is a simple 'example', makes everything public.
you can also do these optimizations:
* inline functions that are used once or twice (no entry in the metadata table + same bytecode -> good compression)
* replace pinvoke calls by the 'calli' opcode
* replace memcpy/memset/... by cpblk/initblk
* replace property getter/setters by functions
* merge classes with only static fields/fns/... into 1
* remove useless attributes
* remove/change names of types/fns/... for better compression
* simplify MD tables: make everyting public, no 'static this'/'extension' methods, avoid generics & inheritance, ...
* use small ldi, br, ... opcodes (like br.s) when possible (dnlib does this for you: linky)
also, for debugging: use *dnSpy* for inspecting .exe/.dlls
again, meet me on IRC :p (#revision on ircnet, #titandemo on efnet, ...)
And on that note: is it really worth the hassle?
plek: if people would think like that, nobody would make demos. so, fuck that. we do it because it's fun and challenging.
Thanks for the lecture on making demos.
Wait there are proper vector/matrix classes in .NET now? Since Framework 4.6 and Core 2.1 already (and the latter even with SIMD intrinsics)? How was I able to miss that? O.O
Let's go on a road trip to Moscow. We'll first drive to Milan, then to Oslo and it only makes sense to then set sail for the final destination.
Sounds like a cool adventure though!
can't argue with that :)
I have create the same test in C++, code size is 29ko compressed with upx... not far from 35ko of C# which uses just zip compression...
porocyon: I have started to look at dnLib, I can enumerate all my methods, but how to know if a method is used by another Methods ? I have just ILCode for each method ...
Another topic... what to use for audio ?
porocyon: I have started to look at dnLib, I can enumerate all my methods, but how to know if a method is used by another Methods ? I have just ILCode for each method ...
Another topic... what to use for audio ?
yes, you'll have to go through all the bytecode of every method. But, the only opcodes that use a method are 'call', 'callvirt', 'calli' and 'ldtoken'.
Also, compare to kkrunchy or crinkler instead of UPX. UPX sucks.
re: audio: ¯\_(ツ)_/¯
Also, compare to kkrunchy or crinkler instead of UPX. UPX sucks.
re: audio: ¯\_(ツ)_/¯
write a soft synth and p/invoke to the waveout api :)
For soft synth with source code :
There is V2 synthesizer with not lot of asm code to convert...
64KLang/4kLang has lot of asm.
Do you know others player ?
There is V2 synthesizer with not lot of asm code to convert...
64KLang/4kLang has lot of asm.
Do you know others player ?