z80 assembler optimizer

Any discussions related to the creation of new hardware or software for the ZX80 or ZX81
santiontanon
Posts: 5
Joined: Mon Mar 08, 2021 4:38 am

z80 assembler optimizer

Post by santiontanon »

Hi everyone, new forum user here!! I have been developing games for MSX and Amstrad CPC for the past few years, and I have been interested in doing something for Sinclair machines for a while! For example, this is my latest MSX game: https://github.com/santiontanon/triton

My latest pet project is an optimizer for Z80 assembler code called MDL ( https://github.com/santiontanon/mdlz80optimizer ). The main idea is the following. Imagine you are working on your game in an assembler file: "mygame.asm", you just do:

Code: Select all

java -jar mdl.jar mygame.asm -ro -po
And MDL will output a list of potential optimizations you can apply to your project to save bytes or CPU cycles. If you want MDL can also apply those optimizations directly, and output an already optimizer assembler file, or even directly the optimized binary.

My main development machine is the MSX, and thus I have targeted MDL to the assemblers that are commonly used in the MSX community (sjasm, asmsx, glass, etc.). By default, MDL will only parse standard zilog notation, but if you want to parse, for example, a sjasm source file, you just add "-dialect sjasm" to the command line, and MDL should be able to parse it.

I would be particularly interested in learning about sinclair projects that I can use to test MDL on (e.g. pointers to github repos), since I have mainly been testing it on MSX projects, and even if (via the "-cpu" flag) it can be configured to assume Z80 settings of Sinclair or Amstrad machines, I have not been able to test this feature very much. Also, open source projects using assembler dialects (e.g. sjasmplus) that are not as common in the MSX world, would be also great for me to be able to test MDL! MDL is still evolving, and it might still have lots of rough edges, but I'm improving it bit by bit :)

Anyway, if anyone is curious and gives MDL a try, any feedback/bug-report is welcome!

(apologies for the very long post for being my first post in the forum, hahaha)
dr beep
Posts: 2060
Joined: Thu Jun 16, 2011 8:35 am
Location: Boxmeer

Re: z80 assembler optimizer

Post by dr beep »

Hello and welcome,

Ambitious project if can really find optimizations.

Standard optimizations like CP 0 vs OR A are quite easy, but what about selfmodifying code for LD A,(NN) vs LD A,0 with pointer to Value A is harder, let alone JR replaced by DB 33 or other opcode when JR is just +1 or +2.

Or exit through other routine with no effect, saving a RET.
santiontanon
Posts: 5
Joined: Mon Mar 08, 2021 4:38 am

Re: z80 assembler optimizer

Post by santiontanon »

Indeed, there's lots of corner cases!! Currently, there are two optimizers:
- the "pattern-based optimizer" (triggered with "-po"),
- and the "code reorganizer" (triggered with "-ro").

The first is a extension of the usual keyhole optimization approach (where there are a set of pre-defined patterns and if any part of your code matches the pattern, then an optimization is proposed). MDL adds a lot of safety checks and constraints (that might have to look at the entire code-base, not just at the piece of code that matches the pattern), to maximize safety of the proposed optimizations. But there's always corner cases where the optimizations proposed my MDL would break the code (e.g. self-modifying code). In these cases you can add a "; mdl:no-opt" comment to the lines to protect them from MDL and ensure it's not going to break your code.

The second optimizer is similar to what the LLVM compiler does. It breaks the code into "blocks" (each block starts with a label, and ends with a jump or a ret), and then tries to re-organize them to eliminate unnecessary jump statements (of course, with some safety constraints, such as not moving code from one page to another, and a few others). Again, there can be corner cases (that can be protected with the "; mdl:no-opt" annotation.

It's still early days, and there are many improvements planned for these two optimizers (and new optimizers coming). I have been mainly testing it with MSX projects (that are usually cartridges, so, no self-modifying code, for example). But I'm very interested to test it in some sinclair/spectrum projects (which I'm sure will surface further corner cases haha). Any pointers to existing spectrum projects I could use to test would be awesome! :)
User avatar
marste
Posts: 250
Joined: Sun Aug 10, 2014 9:58 pm
Location: Italy
Contact:

Re: z80 assembler optimizer

Post by marste »

Ciao Santiago and welcome on the forum!

Your project is very interesting, and I think that despite might not identify extreme optimization cases (and there are cases here where you won't believe what have been done), it might improve a lot of code laying around!

There is somewhere the list of optimizations you implement?

And, I didn't see any flag to choose for speed or size optimizations (that sometime are unfortunately alternative and different optimizations should be applied), or there is?
santiontanon
Posts: 5
Joined: Mon Mar 08, 2021 4:38 am

Re: z80 assembler optimizer

Post by santiontanon »

Thanks!! :)

About the list of optimizations, probably the best way is to see the list of optimization patterns (although those only cover the "pattern-based optimizer") here: https://github.com/santiontanon/mdlz80o ... tterns.txt

I hope the syntax is not too obfuscated :)

Those are the "base" patterns, then if you specify that you want to optimize for speed (with "-po speed"), there's a few extra patterns that are added, and if you specify size (with "-po size"), there's a another small set of patterns added too. I have a few more patterns in my to-do list to add and verify, so the list keeps growing, with the caveat, that the more patterns, the slower the optimizer gets, of course haha.

And indeed, I would not expect any extreme optimizations (hopefully one day ;))! But it's been useful to me at least in saving a many bytes here and there, that otherwise would be wasted hehe.
User avatar
marste
Posts: 250
Joined: Sun Aug 10, 2014 9:58 pm
Location: Italy
Contact:

Re: z80 assembler optimizer

Post by marste »

Very nice parametrization!

Quite understandable, at least to read it! :)

At a first look I found this rule:

pattern: Replace jp ?const1 with jr ?const1
name: jp2jr
tags: cpc
0: jp ?const1
replacement:
0: jr ?const1
constraints:
reachableByJr(0,?const1)

You should put it into "size" optimizations, while into speed optimizations you should put the opposite (transform all jr in jp). Same if there are with flag conditions.

PS: and since sometime they are alternative I think is a good idea to have a default (e.g. size) and an optional speed preference just if specified
santiontanon
Posts: 5
Joined: Mon Mar 08, 2021 4:38 am

Re: z80 assembler optimizer

Post by santiontanon »

Thanks!!! And yes, exactly!!! I have a few of those patterns in the set of patterns for size and speed already (e.g., the one for speed here: https://github.com/santiontanon/mdlz80o ... -speed.txt ). But I am sure I am missing many!

Any more pattern suggestions are appreciated :)
Spinnetti
Posts: 253
Joined: Sat Sep 12, 2020 11:29 pm

Re: z80 assembler optimizer

Post by Spinnetti »

Cool idea, but with a list of patterns we should just be coding that way to begin with - otherwise we never get better!
Zeddy: ZX80, ZX81/ZXpand, TS1000/ZXpand, TS1500/Zxpand+
Speccy: 48k, +, +2, +3, TS2068, "Bare Metal" Pi, Next KS2, IF1/Microdrives/Vdrive/Light Gun/VGA-Joy
QL: Minerva/QL-VGA/Custom PSU
C5: 24v, LiFE battery, Disc brakes
santiontanon
Posts: 5
Joined: Mon Mar 08, 2021 4:38 am

Re: z80 assembler optimizer

Post by santiontanon »

haha, indeed. The way I use the optimizer personally is that I have it integrated into my text editor, so it looks like this:

Image

So, I am coding, and every once in a while, I press the key combination I have associated with it, and it lets me know if there's anything to be optimized. Even having written those rules myself, it's surprising how often we overlook one thing or another (or we just forget haha) :)

That being said, as I mentioned above, the pattern-based optimized is just one of the optimizers MDL has :)
Spinnetti
Posts: 253
Joined: Sat Sep 12, 2020 11:29 pm

Re: z80 assembler optimizer

Post by Spinnetti »

Ah, IC.. cool.
Zeddy: ZX80, ZX81/ZXpand, TS1000/ZXpand, TS1500/Zxpand+
Speccy: 48k, +, +2, +3, TS2068, "Bare Metal" Pi, Next KS2, IF1/Microdrives/Vdrive/Light Gun/VGA-Joy
QL: Minerva/QL-VGA/Custom PSU
C5: 24v, LiFE battery, Disc brakes
Post Reply