Just a quick note to say that we have begun implementing an open-source reimplementation of C64-compatible ROMs, so that we can include them in the MEGA65 without legal complications:
Just a quick note to say that we have begun implementing an open-source reimplementation of C64-compatible ROMs, so that we can include them in the MEGA65 without legal complications:
The MEGA65 has matched the clock speed for some time, but PAL/NTSC mode mis-match can cause the apperance of 1.something or about 0.8MHz.
To be honest, this alternative ROM project is much more interesting to me than the whole Mega65 project (especially that Ultimate64 and C256 Foenix are now available). A pity it stalled as quickly as it started.
Well, it has only stalled in so far as I am not an octopus with as many eyes as tentacles to be able to type on 8 keyboards at a time. I'll very likely attack the next stage when I get a moment, but it is also at a stage that is suitable for the community to join in, and make the whole thing go much faster than if it stays just me alone. There are a bunch of tasks that people could work on, if they have interest:
1. Implement floating-point functions.
2. Implement expression parser (with a nice little cache to speed up frequently referred to constant expressions).
3. Implement variables (with a nice little variable cache to speed up use of frequently referred to variables).
4. Implement missing BASIC commands.
5. Implement missing KERNAL functions.
6. Write lots of tests, and test lots of things!
No idea about MESS, but in XEMU the problem about the CPU speed, that it emulates 65CE02 CPU core with its timing ... more or less. A true C65 would (according to the existing partial documents on this ...) insert "dead cycles" in "slow mode" to more or less match the speed of the 6502 timing. Eg 65CE02 has 1 cycle to execute to NOP, but 6502 requires 2, and so on, regardless of the CPU clock. So I guess what C65 designers wanted is to insert extra cycles in "slow" mode to match the timing of the 6502. For Xemu it didn't worked yet ... For mega65 itself, I have no idea, I guess it works there, and even more work expected (ie giving more 6502 compatibility in "C64 mode" like illegal opcodes). Surely, these can be done in Xemu too, hopefully I get the point some time to do all of these ...
Hi, I'm a programmer interested in computers in so much as I can understand them. So older systems are much more likely to interest me than new ones, which are just really to overblown to get a handle on.
As such I would like to help develop the Free BASIC and Kernal ROMs! For a description of my level of expertise, read on!
I had always wondered how the old 8 bit machines implemented BASIC and have interested in assembly language programming on them as well since the 1980s. But an assembler seemed to be some expensive semi magical piece of technology that was just unaffordable and unapproachable to my younger self. But after studying computer science at university, it's now something that I should be able to do. Theoretically at least.
Once thing I always wanted to implement an assembler on an 8 bit micro back in the day and maybe a high level language. I've studied compiler construction, and did that Nand2Tetris course on Coursera as well
I had been trying to get my dad to dig out ye olde TRS-80 Color Computer II for years without much success, so I started to look into systems that maybe I could buy.
So in the last 6 months or so I started to settle on the Commodore 8 bit micros, looking for a system and typing in a lot of assembler from old books into the VICE emulator running various assemblers I could get my hands on, both in C=64 and C=128 emulation, and also reading and watching all I can to find out how they worked. Including how BASIC was interpreted, tokenised, stored, and ran.
This eventually lead me to actually drop $520AUD on a C=128, 1541 Disc Drive, 1084s monitor and a data set. Which all works great, although the 80 column display doesn't seem to want to sync, though it does display, just the picture keeps rolling and won't lock on, although I'm not sure if that's a problem with the machine or lack of a suitable cable I've tried several stop gap solutions.
However back when it was looking like I might not be able to afford original hardware, I was looking into getting a cheap LatticeXO2 or XO3 FPGA/CPLD board or something like that and developing one bit by bit, no pun intended, starting with say the 6501, then the I/O port to make it a 6510, clocking and PAL, VIC-II, CIAs, SID, etc., etc. until I had something that could run the original ROMs and games. So I poured over all the relevant data sheets and started learning VHDL...
I think I have a pretty reasonable understanding of how the Commodore 64 actually works now, Commodore 128 a bit less.
After I got the Commodore 128, I was still a bit interested in the FPGA stuff, and thought probably the best thing to do was try to create something simple like an REU, then say a SuperCPU, then a C=64, then a C=128.
But then I saw the talk Paul gave about the FPGA phone in Christchurch in January <https://youtu.be/KuNB4ocZDXA> and saw what I wanted to do had essentially be done, and so I want to contribute! It seems a better use of my time that writing yet another assembler for the C=64 or C=128, and I'm fresh out of ideas for games, etc...
The 'cmd_load.s' contains:
;; XXX - C64 BASIC apparently doesn't clear variables after a LOAD in the
;; middle of a program. For safety, we do.
This will break some BASIC software, like:
100 DN = 8: REM set device number
110 LOAD "SPRITE DATA 1", DN, 1
120 LOAD "SPRITE DATA 2", DN, 1: REM <- this will try to load from device #0
It would be great for you to help out on the open-roms, if you are willing. Also glad you enjoyed the talk at LCA :)
Anyway, the source is at https://github.com/mega65/open-roms, if you haven't already found it.
If you were willing to start with implementing floating point format handling, that would be great, as together with the expression parser it is the key to getting the BASIC really useful. It's important that we don't copy the C= code, but implement everything fresh, so I would even recommend not reading up on the algorithm that they used, but do it from scratch how you think best. I'd begin with floating point number parsing, and then move onto displaying floating point numbers, and performing calculations on them. There are a bunch of good academic papers on parsing floating point numbers in recent times, because floating point is the default data type for many modern scripting languages, so a lot of work has been done on improving the algorithms for working with them.
The 'LICENSE' contains:
This software is Copyright Paul Gardner-Stephen (2019). All rights reserved.
It must not be used or distributed without prior written permission of the author.
NOTE: This is a placeholder statement until a final license is selected.
Somehow, I don't feel comfortable trying to use this...
First, thanks for the heads up on the behaviour of the LOAD command. Can you please create an issue on github for it?
Now, for the license, I think I have a pretty good record of putting things under appropriate open-source licenses, so I wouldn't worry. The reason I hadn't chosen one, is that I wanted to talk with the community about what the preferred license would be. My gut feeling is LGPL, but I would like to hear what other people think. Then I'm totally happy to change the license file to indicate what we settle on.
Issue created. How do I clone the whole repository to build the ROMs? 'git clone' worked, 'git submodule init' too, but 'git submodule update' gives me:
email@example.com: Permission denied (publickey).
fatal: Could not read from remote repository.
 Found a [solution](https://github.com/desktop/desktop/issues/6573#issuecomment-453158514)
Hey, thanks was thinking of starting there anyway, floating point parsing, manipulation and representation, probably working backwards so representation and manipulation followed by parsing.
But I was a bit taken aback by your reply and what I read when I read your more of your blog. I see that you aren't following the same tokeniser scheme that the C=64 did.
Fair enough I guess, would be easier to implement a better symbol table that way. For instance say keeping the identifiers heap sorted so lookup of any symbol is O(log n) where n is the number of symbols.
I guess I'll see when I look at the code. I'm guessing you've got something working anyway, considering you can do LOAD, etc.
But I'm curious as to whether you want to keep the same Microsoft floating point format that the C=64 uses and all the memory addresses in the zero page the same way that the C=64 does as documented in _Mapping the Commodore 64,_ etc. As far as I'm aware doing so would be crucial to getting binary compatibility with existing software that used the KERNAL. I'm guessing the tokeniser format not so much.
How to do the transcendental functions is a bit of a vexed question. There's basically two ways to do it I know of. The first is polynomial evaluation of something close to the Taylor series but with the constants of the polynomial finagled slightly to give more accurate results than using only the first few terms of the Taylor series and ignoring the rest.
Terms which grow increasingly smaller, but increase in order and are infinite in number. And which unfortunately don't settle down to "negligible" for quite some number of terms. So (empirically derived?) correction is added to the first few terms to compensate.
This is how the C=64 did it, and in fact if you are keeping binary compatibility with the BASIC and KERNAL ROMs you actually have to implement this polynomial evaluator because it's exposed and documented so presumably anyone could have used it.
Then you might as well use it to evaluate your transcendentals, but then you are in the position where the table of constants of your polynomials would be the same as the copyrighted Microsoft floating point implementation.
Now I doubt this would be copyright-able, given that to get the same results on the same data (achieve binary compatibility), you are basically stuck with using these constants. So presumably they would be more "facts" than "inventions" or other protectable works, but I'm not a lawyer. And it generally seems against the approach that you want to take.
The other approach would be to use a CORDIC algorithm to do transcendentals. This might even be faster and would not be based on anything Microsoft or CBM did. Unfortunately I don't understand CORDIC yet :P
__...and would not be based on anything Microsoft or CBM did__
I misspoke. What I should have said is "it would then use a completely different approach to computing transcendental functions than Microsoft used, so there would be no question of copyright infringement".
Yes, my feeling is that keeping binary format compatibility will be important. One of the nice things about building the ROMs progressively, is that we can change our design decisions as we go along.
My view is that we should always minimise risk of copyright infringement first, and then improve compatibility after. In particular, any forced convergence with the original ROM should be supported by an issue that demontrates a program that cannot run without the convergence, e.g., in this case find a program that relies on the data format of the floating point numbers to work, and in the meantime, implement it however we think is best practice. What is certain, is that the most of the implementation will be re-usable, even if we started with a different format.
But otherwise, it is really encouraging to see the interest from yourselves in helping out with this project :)
Ok, I think I have warped my tiny brain enough so CORDIC can fit in :D At least for computing sine and cosine...
By the way, I cloned the repo, but Paul do you have a link to any documentation about what the work flow is? What assemblers, etc.
Also I think we probably need to start noting down what volatile memory addresses are used and where. Because obviously we can't use locations that are available to the programmer, because then they aren't available to the programmer anymore! Not a huge deal with functions that don't call any other, so long as certain memory addresses are reserved for these leaf functions, but even there we still have to worry about what locations might be trampled by an interrupt handler.
_Mapping the Commodore 64,_ _Mapping the Commodore 128,_ and the _Commodore 64 Programmer's Reference Guide,_ etc., can help but only so much. We can use it to know what locations are used by BASIC and KERNAL but not really to stop one part of the OS rofl stomping another part, written by someone else.
The stack isn't as helpful as it could be 1. because stack relative addressing isn't directly supported on a 6502, although it can be arranged by getting the stack pointer into X with TSX and then using relative addressing eg. LDA
That doesn't help much though because the stack is only 256 bytes! So facilities for recursion would seem to be quite limited, so every bit of data will have to be a global pretty much, and pretty much either located by prior agreement or doled out by central mechanism in the KERNAL..
This could be simple as a free list but even still, it's still a pain because were do you keep that address you receive. Putting a couple of them on the stack isn't likely to be catastrophic, but otherwise you're back to the chicken and egg problem of you have to store it somewhere, and that requires an agreed upon location...
“I'm guessing you've got something working anyway, considering you can do LOAD”
I can't get this to work - it's probably incomplete, as even CIOUT is clearly not implemented. I'm going to take a look into this.
“What assemblers, etc.”
It uses Ophis assembler, it's in the submodule. Under Linux typing 'make' in the 'open-rom' directory did the trick for me (just one REALLY nasty thing: I had to switch Python to version 2 :/)
@gardners I've forked the project and started working on a DOS wedge:
If I'm doing something really ugly, please tell me. I didn't touch 6510 assembler since my childhood. Plan: use it as a simple testing tool to fill-in gaps in the current Kernal IEC support.
One question: are you planning any kind of compile-time configuration? Like: FEATURE_DOS_WEDGE=Y somewhere... I try to optimize code for size, but we might run out of ROM space at some point.
I'm planning to add a pre-processor, so that we can have #ifdef and friends for conditional compilation. One added trick is that we may want to select whole files on the basis of such things, so I'll probably add some facility for indicating inside a file which targets it is/isn't meant for, or which optional features it is/isn't needed for.
As for CIOUT, the necessary code is implenented in LOAD, but hasn't been broken out into all of the necessary separate routines. You are welcome to do so, it shouldn't be too nasty a job, and it will be very helpful. Perhaps start by making the issue on github first, and then you can reference the issue in the commits as you refactor, so that we can trace what is going on?
I've just thought of a potential show stopper: If LOAD works properly, shouldn't it just load the data in the file to the address shown in the header? If that is the case, then clearly it isn't going to work with saved BASIC programs, which if memory serves are stored as data as they appear in memory. Although it will work just fine for assembly language programs.
^are store as they appear in memory, already tokenised
I meant to say, it's not going to be able to LOAD BASIC programs written on any other machine or indeed KERNAL and BASIC ROM set than the one being developed, because it doesn't do any interpretation or tokenisation, it just places the bytes it sees after the two byte header, one after the other, to the address in memory as given by the little endian memory address stored in that header.
That works for both machine language programs, BASIC and even things like sprites, because when a SAVE is done, it just defaults to saving the BASIC area of memory, where the commands are stored. But you also instruct it to save a different area of memory. In either case, this starting address is saved as a two byte header so it can be LOADED back, raw, into the memory address it came from.
It will take me a couple of days before I reach CIOUT. I also need SETNAM (not yet implemented), CLALL would be nice to have (I call it at the start of each @-command so that I'm sure I have free channels for opening), I need vectors under $031A (I want to be system friendly with my wedge, so that it has a chance to work with IDE64, RAMDOS, etc.), and who knows what more - well, even more 6502 assembler knowledge (I was never really good at it - I had very little literature on the topic, and no Turbo Assembler, only FC III and AR carts back in the day...).
I think I'll just implement/test what's missing and then we will refactor the LOAD.
Paul at some do you want to come on to the Discord server that Terlisimo has set up to have a quick chat about things please? Thanks
@Goatflakes - tokenization result has to be compatible between original ROM and this implementation, there is no other option.
LOAD "filename",8,1 - loads data from disk to the location stored in the file (2 byte header)
LOAD "filename", 8 - loads to the default location (on C64 standard is $0801, but this can be changed by modifying TEXTTAB variable in zero page), afterwards the pointers within BASIC program (single-linked list) are recreated, the 2 byte header is ignored in this case
We have another problem: autostart software. It typically loads from well below $0400, overrides various vectors (so if ours are different than original ones, we will end with crash; that's the reason why autostart software disables most of the BASIC extensions, etc.). And we can't just ignore the bytes overriding vectors, as sometimes the autostart software contains slightly modified vectors, to point it's routines :/
But my biggest concern are all these '*_under_rom' routines, which are placed in memory area reserved for use by the user, this is a disaster waiting to happen. IMHO the concept will have to be changed at some point, for example the peek could work in the following way:
- peek from below BASIC ROM or in the $C000-$CFFF area - operate normally, o bank switching, no interrupt handling, etc.
- peek from RAM under BASIC ROM ($A000-$BFFF) - handled by code located in KERNAL, should temporary disable BASIC ROM to perform the peek, no interrupt handling necessary
- peek from RAM under KERNAL ROM or IO area ($D000-$FFFF) - code located in BASIC ROM should temporary disable KERNAL ROM to perform the peek, only this part should do the interrupt handling workaround
If something is written to the address of ROM, isn't it written ok, but a read will get the value in ROM until location 1 is changed to switch out the ROM, except if you configure the VIC-II to read from an address there, it always gets what's in RAM. Or at least that's how I understand it :P
We should make the _under_rom stuff optional via a #define / compilation option, so that it is easy to generate 38911 bytes free target where compatibility is more important, and the 60KB free variants for folks who want a better basic. Again, the joy of open-source is that we can make it easy for people to pick what they want.
@gardens, I've got some questions:
1. Did you actually test the 'iec_tx_byte' and 'iec_rx_byte' routines?
They look complete to me, but... Kernal 'open' routine also looked complete, but now I see that it requires some work - for example, support for the LAT / FAT / SAT tables is completely missing (I'm going to add it now). So far my work is 80% research, 5% writing code, 15% testing with 'printf' and Vice built-in monitor :)
2. What will be your policy regarding merge requests? Do you expect me to run the similarity tool and add comments, or would you prefer to do it by yourself?
3. Could we finally settle the license? LGPL is fine for me. If you are unsure, you can add a contribution policy - that we all allow you to release the code (with our contributions) under any OSI-aproved open-source license; that's also fine for me. But I would really prefer to have some proper license.
1. Yes, in so far as I got it to the point where I could LOAD programs. Note that VICE tries to get all clever pants and intercept points in the KERNAL for disk access unless you tell it not to, and of course it assumes it is patching a stock KERNAL. It would be great for someone to offer a patch for VICE to make it only do it when it detects an original ROM. And, yes, I realise that there is a lot of the finer points of the functions already written that need to be implemented, and I am glad that you are thinking about them. The limited time I had on hand to boot-strap this project meant that it just wasn't possible for me to do everything, but this is also why I am so glad that it is now no longer just me working on it :)
2. Hmm.. This is a question I had not come to a firm answer on. The overall issue is to avoid any risk of copyright infringement. My current feeling is that a single pull-request each time, that includes the similarity strings documented, and perhaps with added documentation about the context in which they occurred that allows us to more rationally assess each contribution to the code base. It's a pain that we have to do this, but I think it is for the best. On that note, it probably makes sense for us to make a tool that allows us to show the similarities in context in both the original and our ROMs, so that we can argue for the reasonableness of each. We basically should have a "copyright infringement avoidance" review on every contribution from here in, including any further work that I do. How does that all sound?
3. I completely agree and understand. As you and I are currently the complete set of contributors at the moment, and we are agreeance, we shall deem it to be the policy. I'll update the license file. I've forgotten that organisation that makes contributor agreements for projects, but we should use their thing to make one and include it in the project. Would you be willing to hunt that side of things down for me?
Your friendly neighbourhood moderators: Deft, gardners, Ralph Egas