pouët.net

64k intro in c#

category: code [glöplog]
Someone had developed 64k intro using .net framework ?
added on the 2018-05-04 21:07:48 by fxgen fxgen
I don't think this is viable/possible at all. Please correct me if I'm wrong.
added on the 2018-05-04 21:15:29 by jco jco
I think it would be tough; executable compressors do exist, but I'm not sure their compression ratio vs the .NET code size makes it viable.
added on the 2018-05-04 21:29:52 by Gargaj Gargaj
I *think* it's possible (I have been thinking about this for quite a while), but it'll need some effort.

.NETZ seems to just compress the binary and add some hacks to make it work, like a basic dropper. This is far from ideal, as the file format is rather redundant and stupid (UTF-16 strings, really?).

.NET binaries consist of the bytecode (which could benefit from a transformation à la kkrunchy), and the metadata tables (lots of uninteresting stuff you probably want to get rid of, unless you're using reflection). Libraries such as dnlib can be used for this. (Mono.Cecil only performs 'high-level' bytecode modification, so it isn't suited for this.)

There are multiple ways to approach this:
* 'nondestructive transformation' of the binary: make the data more compressible while leaving it legible for the parser, then prepend a smaller CorExeMain stub.
* Store the data in a custom format, reconstruct a valid file at runtime, then execute it.
* This can be done using both a .NET dropper or a native dropper. Both have up- and downsides. (Cross-platform, code can be loaded through reflection, but it's large, vs. small, but harder to set up, and both the .NET Framework and Mono have to be taken into account.) (Though, I'm probably the only one here who cares about Mono.)
* Compile from source at runtime. Most .NET installations come with csc (command-line C# compiler) automatically, but I don't think this is the case for newer .NET versions, although I'm not sure (it's been ages since I last used Windows). Most Linux distros (and probably BSDs, too) ship the Mono runtime and the entire thing separately as well.

Of course, a compressor has to be fit somewhere in between, and getting a window + an OpenGL context set up probably won't be easy either, as XNA/MonoGame/OpenTK/... won't be available. (You probably want runtime code generation to create all the OpenGL functions. Either do this in the dropper, or using System.Reflection.Emit)
added on the 2018-05-05 16:54:34 by porocyon porocyon
I have done a test with a c# launcher that decompress ilcode (intro part) and run it.
I can have an executable <64k with a 3d metaball scene using opengl ...
Compression/Decompression is done using gzip from .net framework.
Problem is to remove automatically unless classes and methods.
That a start.
added on the 2019-11-14 21:53:26 by fxgen fxgen
you could try to use Unity's il2cpp tool which is how they are able to ship unity games on iOS, Switch, etc...

Here's a 4 year old blog post

il2cpp

Whether or not you can get close to 64k I have no idea. I'd try just "Hello World" to see if it generates something small or not
added on the 2019-11-15 06:04:08 by greggman greggman
Or let's have more 128k or 256kb intro compos at parties.

Would be interesting to see battles between C# (or other language frameworks) and native c++ intros.

And larger filesize would possibly allow more interesting content (more sound samples, compressed meshes, etc.)
added on the 2019-11-15 10:15:35 by arm1n arm1n
Good thread. C# is my favourite programming language.
added on the 2019-11-15 11:41:50 by Adok Adok
Bad post. C# is not my favourite programming language.
added on the 2019-11-15 11:49:42 by arm1n arm1n
fxgen: hmm, that's interesting. I remember writing a tool that removes unreferenced metadata table entries from an assembly (in F#, because reasons), but idk where it is, and how well it worked. Though I suppose it isn't too difficult to roll your own, by using something like dnlib (which is more low-level than mono.cecil), as it's basically just graph traversal. you can then also change/zero out the names of the symbols (while leaving the references themselves in place properly), clear as many flags as possible to decrease entropy, etc etc.

I'm a bit too lazy to start hacking on a PoC, but you can always hit me up on IRC :p

greggman: that'd be cheating :p (make the same binary work on mono as well, I dare you.) plus, iirc, you need the paid version for this, which, double ew.
added on the 2019-11-15 12:12:11 by porocyon porocyon
Quote:
Compile from source at runtime. Most .NET installations come with csc (command-line C# compiler) automatically, but I don't think this is the case for newer .NET versions, although I'm not sure


FWIW, the Windows 10 installations I have flying around here all ship with

Code:%windir%\Microsoft.NET\Framework64\v4.0.30319\csc.exe


I don't know how useful this is though. The compiler version itself is 4.8.3752.0 (on both 1903 and 1909), but it says it can only do C# up to version 5.
added on the 2019-11-15 15:14:05 by KeyJ KeyJ
when using "System.Numerics" for vectors and matrices stuffs I have an 35 ko executable size ! I have my own class for maths before...


Porocyon: how to use dnlib to find unless methods ? Thanks.
added on the 2019-11-18 14:01:17 by fxgen fxgen
fxgen: basically,
1. find the entrypoint
2. in the entrypoint: for every local variable type, and for every instruction that uses a function/constructor call, property getter/setter, field, ..., add that function/ctor/field/getter/setter to a list, and add returntype/argtypes/... + the type that contains the fn/ctor/... to that list too (you can also use one list for fns, one for types, etc etc)
3. do n° 2 for every fn/ctor/getter/setter that was added to the list
4. GOTO 3 while you keep finding new fns/ctors/...
5. for every type/fn/ctor/... in the assembly: if it is NOT in the list, remove it from the assembly

if you want to see code that uses dnlib: a long time ago I used dnlib to inject code into Terraria, for a modloader, linky. 90% of it is MSIL patching and not modifying types, but here is a simple 'example', makes everything public.

you can also do these optimizations:
* inline functions that are used once or twice (no entry in the metadata table + same bytecode -> good compression)
* replace pinvoke calls by the 'calli' opcode
* replace memcpy/memset/... by cpblk/initblk
* replace property getter/setters by functions
* merge classes with only static fields/fns/... into 1
* remove useless attributes
* remove/change names of types/fns/... for better compression
* simplify MD tables: make everyting public, no 'static this'/'extension' methods, avoid generics & inheritance, ...
* use small ldi, br, ... opcodes (like br.s) when possible (dnlib does this for you: linky)

also, for debugging: use *dnSpy* for inspecting .exe/.dlls

again, meet me on IRC :p (#revision on ircnet, #titandemo on efnet, ...)
added on the 2019-11-18 15:37:06 by porocyon porocyon
And on that note: is it really worth the hassle?
added on the 2019-11-18 15:52:07 by superplek superplek
plek: if people would think like that, nobody would make demos. so, fuck that. we do it because it's fun and challenging.
added on the 2019-11-18 17:21:39 by porocyon porocyon
Thanks for the lecture on making demos.
added on the 2019-11-18 17:29:40 by superplek superplek
Wait there are proper vector/matrix classes in .NET now? Since Framework 4.6 and Core 2.1 already (and the latter even with SIMD intrinsics)? How was I able to miss that? O.O
added on the 2019-11-18 17:33:20 by kb_ kb_
Let's go on a road trip to Moscow. We'll first drive to Milan, then to Oslo and it only makes sense to then set sail for the final destination.
added on the 2019-11-18 19:40:58 by superplek superplek
Sounds like a cool adventure though!
added on the 2019-11-20 20:29:55 by ton ton
can't argue with that :)
added on the 2019-11-21 15:26:26 by superplek superplek
I have create the same test in C++, code size is 29ko compressed with upx... not far from 35ko of C# which uses just zip compression...

porocyon: I have started to look at dnLib, I can enumerate all my methods, but how to know if a method is used by another Methods ? I have just ILCode for each method ...

Another topic... what to use for audio ?
added on the 2019-11-22 23:07:12 by fxgen fxgen
yes, you'll have to go through all the bytecode of every method. But, the only opcodes that use a method are 'call', 'callvirt', 'calli' and 'ldtoken'.

Also, compare to kkrunchy or crinkler instead of UPX. UPX sucks.

re: audio: ¯\_(ツ)_/¯
added on the 2019-11-22 23:24:33 by porocyon porocyon
write a soft synth and p/invoke to the waveout api :)
added on the 2019-11-23 02:05:23 by xTr1m xTr1m
For soft synth with source code :

There is V2 synthesizer with not lot of asm code to convert...

64KLang/4kLang has lot of asm.

Do you know others player ?
added on the 2019-11-23 14:36:53 by fxgen fxgen

login