Sunday, 17 June 2018

"Data East Hits" 8-bit Gaming My Arcade Pixel Classic mini console


Disclaimer: Your kid might in actual fact eventually become bored
Over the years I've seen a bunch of "50 games" type handhelds at supermarkets or department stores. As the games and physical design are always so bland and generic, I've avoided them.

Now I was lured in by a Data East-branded "308 games in one" cheap device that comes in various attractive shapes, notably the horizontal Lynx/Gameboy Advance form and the more obviously Game Boy shaped hand held console. There's also a mini arcade shape that I did not look at twice.

Options. Oh the buttons can be lit up too. Why?
I went for the Gameboy shape for novelty's sake, although to be honest the horizontal form might be better as a controller and for connecting the leads.

Ergonomics to hell, the case has been shaped with "pixel arty" corners, but this matters very little. Designers take note, perhaps this is an indication we've got past that roundening-era of design styling.

The plastic feels a bit flimsy, but the buttons give an adequate response. There's a 5V micro-USB style power connector, AUX out and AV out. The device works on 4 AAA batteries, but the other options sort of sold this to me, even though no cables or batteries were included with the price. (about 30€)


The game collection

The Data East name is boldly visible on the package, and as their back catalog boasts some really iconic stuff (Burger Time and Bad Dudes vs. Dragonninja) I was prepared to expect at least a good bunch of playable games.

I prefer Mr. Do's Castle (not included)
The word "arcade" might mislead you to think these are arcade version of the games, not to speak of the screenshots in the inlay card.

I can't believe that in this day and age I could still get duped by a "wrong platform screenshot". Thanks for making me feel like a child again, but perhaps not the way I expected...!

Screenshots may vary.
Yet the shots are made in a way that although built from arcade screens they might arguably represent "game covers" rather than actual gameplay, hence the logo overlays. And it DOES say on the cover, "8 Data East + 300 games", but that "plus" is also quite easy to miss. And it DOES say "8-bit" which should clue in that the games can't be arcade quality.

Let's make it clear: About 95% of the games are simplistic, NES bootlegs or otherwise very poor quality.

Real screenshot for comparison
On the first page we get Dragon Ninja, Break Thru, Caveman Ninja, Heavy Barrel, Side Pocket, B-Wings, Karate Champ, Burger Time... These are up to the quality of the NES versions of these games, which is to say they are not all that hot but mostly they play well enough.

But after the first couple of pages it goes downhill. So, enjoy the excitement of iconic 8-bit games the likes of Man in Red, Repair Urgently, Speed Man, Unusual Space, Girl, Tactful Monkey, Devildom Doom and so on!

There's pages and pages of this shift
Many of the games are old bootlegs and hacks of licensed games, like Dada is obviously Popeye, Primitive Woman is supposedly a Tom and Jerry game and Sunken Ship is worked from 2005's Titanic, which in turn is a hack of a Tom and Jerry again. Astro Robo To To is Astro Robo Sasa. Aether Kadass is Macross. Sky Invader is Sky Destroyer. These bootlegs can be semi-playable or interesting as such because the game content is kind of there. But I'm wondering how this particular collection of games got to be curated.

There are many scrolling vertical shooters, the best might be F22, with actual power ups, boss fights and varying levels. I guess it is another rework of something else.

The bootleg of a bootleg
I'm prepared to say some might even be filler crap made specifically for this device, varying a base source code to produce simple shooters and jump'n'runners.

Some games have visuals that go beyond NES, like Cut Fruit and Curly Monkey 2, but design-wise, there's not much thought on the gameplay or level design.

Some games are so simple it's harder to fault for what they are, like Horse Racing where you simply duck and jump with the horse. However there's a bunch of boring sprite-shoots-sprite stuff, guy jumping on platforms, or collects stuff falling from heaven, with slight variations and different skins. There's even a digital paper/rock/scissors game, which must be the pinnacle of sadness.

Dark Castle. Wasn't this Thunder Castle on the Intellivision? Looks crappier.
At least one track'n'field-type game has been blown up into a dozen, with javelin, long jump, hurdles etc., even making Breast Stroke and Butterfly Stroke into separate games. Although I have to admit it may be a better idea than try to endure a full decathlon. A panda takes the place of the sportsman, possibly a clue about the country of origin of these games.

Then there are card games and board games such as mah jong, reversi, checkers, sudoku and so on. These can't be overtly blamed if they are simply bootlegs of once commercial games. Sadly there's no chess, though. Other genres are absent too, there's no Tetris or Columns clone for example, and no proper top-down driving game with rotational controls either.

Dark square at right = Portuguese draughts? Your move.
Then there are doubles, that is the same game with different graphics. This is a bit irksome as usually both graphic variants are just as visually uninspiring.

Overall the games boast very mediocre, bland and unimaginative visuals and sounds, while the game titles convey an idea of non-IP-infringing nothingness. It's almost as if I'm witnessing a relic from an alternative universe Earth culture that had strangely failed to invent any of the central video game brands we have.

You go... eh, girl!
All in all, the Data East games are probably the most playable of the bunch and Burger Time might indeed be the best game on the entire device.

I'll review all the 308 games later.


Final verdict

I took the device to a composite video TV. As I don't have a proper 2,5mm AV cable I used a single tip/sleeve version which produced only a black and white image. Or is the output only B/W? This was enough to show that the device does produce a smooth frame rate for the TV, which is nice. Of course a few of the games give jerky scrolling or flickering sprites due to programming techniques.
A 5V 1A phone recharger cable seems to be sufficient for powering the device (no specs in the manual) so the absence of a cable in the package might be justified. However the 2.5mm TV cables are less often seen.

One annoying thing is the device can't remember the volume setting when returning to the menu. it can remember the menu positions so why it can't keep silent? Wading through the 308 game menu can be tiresome, even if it can be scrolled sideways too.

So, is the My Arcade 8-bit Gaming Pixel Classic any kind of value for money? Despite all the misguidance, I could still be generous and say there are about 30 reasonable quality games in it. Even then 1€ for each might still be a bit too much!

It's a bit sad considering the hardware might be capable of doing justice to the arcade versions of the games, but the conversion job would have been overbearing. I doubt the device is good enough to really emulate the suggested arcade platforms.

Perhaps if the game ROMs can somehow be rewritten, the device might be much more interesting because the physical hardware is OK-ish and the screen and sound quality is not that bad. I am a bit doubtful if anything inside can be easily upgraded, though. Missed potential I'd say.


Addendum 18.6.2018

As I saw the device might be easily opened, I had a peek inside:


For those interested in subduing the sound, this might be achieved by physically blocking the loudspeaker a bit more. (There are like 2 loud sound levels + silent)


The parts are not especially neatly connected so it was a bit of a bitch to put back together. So I'm probably not going to open it again.


The game ROMs are likely buried inside that alien tar shit thing. (Hmm the SPANSION S29GL256P10TF is memory, though). Putting the screen back, it has to be straightened by hand. Luckily it can be done after the board has been screwed back in.


Monday, 11 June 2018

Fixing a Fostex 280



I have had a Fostex 280 (1989) 4-track tape recorder/mixer for about 20 years. I bought it used, around 1998, did some funny stuff with it but then generally forgot about it. At some point I considered it a junk item as the tape recorder had ceased working.

Now I found a 4-track tape with something I'd not heard in 20 years, which gave a motive to look at the device again. Searching the net I was pretty sure the tape recorder belt was faulty, which was also easy to confirm.

Also, I had probably misused the multitracker back in the day, so it might be interesting to record something with a bit better understanding of sound dynamics and a more appropriate cassette type.

The Fostex does 4-track on a C-sized cassette, simply creating four magnetic trails instead of two, using a higher speed than ordinary tapes. The mixer part has 4 main channels with equalization, gain, two AUX buses and various re-routing options. There are also 4 more channels with less options, giving a total of 8.

Channels can be recorded on one track, or stereo-recorded over two tracks at once. Tracks can also be bounced, re-recording for example three tracks onto one, giving more space for new tracks.

The digital control over the 4-tracker is pretty neat, as the tape counter can be set to zero at a convenient point,  and then >>0<< rewinds the tape back to that position. There's also one alternate memorized position that can be used as a loop end position, facilitating loop and auto-record.

Looking at eBay listings, a working Fostex 280, although not super-expensive, is not exactly trash either. Going back the memory lane, I remembered how fun and uncomplicated it was to do a recording without a computer.


Replacing the belt

As I saw the multitracker was not completely lost, becoming motivated again, I then ordered a replacement belt from eBay and received it in two days. Day later, I had it replaced.

When the belt has disintegrated, the tape deck can't really behave at all. As the Fostex is switched on, it will give a horrible creaking sound and an "E" on the display. Furthermore, the tape head may be left awkwardly in a position that makes it difficult to remove or insert the casing cover or even a cassette.

The dreaded "Red E of Death"
These are often indicators of a broken belt and replacing the belt got rid of these problems. Listening to my tape was bit of an anticlimax though, but nice experience in a time travel sort of way.

This video by the Youtuber account "3rd island" shows absolutely everything about the process. Without it I might have wasted time or even broken things. I'll recap the main points below:
  • Remove all faders, unscrew the case cover and then gently pull out the cover.
  • Disconnect cable connectors that might hinder the cover. There's no point in trying to keep the different units connected during repair, if they are in your way. The connectors are quite well color-coded and have logical sizes, but just to be sure you might want to photograph the starting position.
  • Pull out the circuit board with the faders. There's a foil between this and the circuit board below. The foil does not need to be fully removed, just disconnect all the connectors that lead to the tape unit, and make some note of their routes between the capacitors.
The white, red, blue and yellow are for the tape unit.
  • Remove the four screws that keep the tape unit in place. Take note of the ground wire that is also screwed in. Pull out the tape unit.
  • A metal panel needs to be removed before the belt can be inserted. This part is held to the unit with only one mechanical screw. Take care with removing the screw as this holds a couple of other parts in too, mainly the spring. Also, take at least a mental note of how the spring is held in place, before loosening any parts.
  • Now the belt can be positioned around the metal wheel. The other end may be stretched around the small black plastic protrusion next to the plastic wheel, which is the real target for the wheel. Also, make sure the plastic wheel and the metal wheel are clean.
The belt is temporarily stretched over the black plastic stick.
  • Putting the tape unit back together, make sure the spring is in correct position. After the screw is in, the belt can be moved from the temporary position to the actual wheel. As shown in the video, tweezers are good for moving the belt.
  • The ends of the spring need to be re-located, use a combination of a small screwdriver and tweezers to achieve this.
 Left: The hook for the top side of the spring end (not in place). Right: bottom spring end hook (in place)
  • Put it all back together, don't forget the grounded wire under one of the four screws and don't forget any of the connectors.

What makes this fairly simple is that the insides of the Fostex are quite modular and the tape unit is easy to remove.

As I went through the process I soldered back one of the headphone sockets which had come loose. To tell the truth there's a lot of other problems with this multitracker, such as a bit lazy/random rewinding and fast forwarding. Some of the channels have ceased working properly.

I'm wondering if some of these might relate to bad capacitors or similar issues. So, the recording experiments may have to wait for a while. But at least I got my sound out from the 4-track tapes which was enough for now.

Sunday, 27 May 2018

QLDD

With my Sinclair QL now armed with a disk interface and HxC Floppy Emulator, I wanted a simple way to access a Double Density disk image format and adding/removing files to it.

The qltools.c can be compiled on linux gcc, but I had a feeling I could also work towards understanding the raw image format a bit more deeply, and then write the HxC Floppy Emulator (HFE) files myself.

This is not really a blog post to be read, these are mostly very long "notes to self" about the topics that were needed to achieve what I wanted.

I'll look at:

-The Floppy Emulator 737280 raw byte disk image format and the QL disk ordering
-The Floppy Emulator HFE bit-level disk image format (MFM)
-CRC calculation

Bear in mind I'm not explaining IMGs and HFEs fully, I'm just doing the minimum to manipulate files on a 720k QL DD disk image.

For understanding the QL disk format, the below sites were valuable, but even then I had some head-scratching to do.

About the sector ordering:
https://soulsphere.org/hacks/ql/fs.html

Some useful disk terminology from here:
http://www.zen35309.zen.co.uk/misc/wxqt2.html

More definite header information from here:
http://qdosmsq.dunbar-it.co.uk/doku.php?id=qdosmsq:fs:dsdd

The QLTools source code:
ftp://ftp.nvg.ntnu.no/pub/sinclair/mirrors/ql/maya/qltools/vers_2.01/qltools.c

I am looking at a QL Double Density drive raw byte image as a middle-ground for transferring between PC/Linux files and the SD card-friendly HFE image. The image I'm working with is 80 Tracks, 9 disk sectors, two sides, with a sector size of 512 makes 737280 bytes.


A Block is a central unit here. The Block comprises of three sectors, making up 512 * 3 bytes, i.e. 1536 bytes, and there are 480 of these Blocks on this DD disk.

Two Blocks are already taken up by the disk header and File Allocation Table (FAT) itself and the Directory File.

Each QL stored file takes a minimum of 1 Block of space on the disk, even if I saved just one byte. Also, each QL file is preceded by 64 bytes of data, so a file saved on QL Basic...

SBYTES flp1_test, 196608, 1500

...will take two Blocks, as the true length of the file with the information will be 1500 + 64 = 1564, more than 1536 and no longer fits into one block. Additionally, the file takes a 64 bytes slot in the already reserved directory list block.


The raw byte IMG file

I will first look at a stored raw IMG file, converted from a HFE using the HxC floppy emulator utility software.

The raw IMG is exactly 737280 bytes long, so within the IMG there are no additional headers or other data than the floppy image as a byte representation form. Contrast this with the floppy emulator HFE image, which represents the bits on the actual disk and also has a header that describes the disk format.

For purposes of analyzing the image, I found it useful to display the IMG contents as hex/ASCII with 512-wide rows. The below shows the beginning of the DD image:

(Note that the block/sector order does not continue as neatly)
The only really certain thing is that the first 95 bytes are the header.

I guess the look-up table inside this area should describe the order of the sectors, but I use a fixed format/order.

Within this fixed format, the first block will contain both the header and the File Allocation Table, beginning from offset 96. The table is 12-bit, although I'm not sure that it corresponds to any FAT-12 standard.

What I call the "directory file header" in the above image is really one file taking up one slot in the directory list file itself.

The FAT contains 480 three-byte records, corresponding with the 480 Blocks that are on the disk of this size. After the FAT the image will contain by default one directory data Block, at block 1, and after this it's just empty blocks to the end of the image.


IMG File Allocation Table

So, in this disk the first block containing the disk header and the FAT information can be stored to a linear buffer by appending 512 (0x200) bytes from 0x0000, 0x600 and 0xC00 offsets each.

The FAT is made of 480 three-byte allocation records, and a record is made of two 12-bit values:

File Number 0x000 to 0xFFF
Sequential Number 0x000 to 0xFFF

Looking the record as three bytes, it is constructed this way:

Record #n
byte 0: 8 Most significant bits of file# value
byte 1: 4 Least significant bits of file# value, 4 Most significant bits of seq# value
byte 2: 8 Least significant bits of seq# value

An unused (empty, free) record has the values:

0xFD, 0xFF, 0xFF

e.g. File number 1, sequence 0:
0x00, 0x10, 0x00

e.g. File number 1, sequence 1:
0x00, 0x10, 0x01

0xF8 indicates that the block is reserved for the header/FAT. File 0x00 is commonly the directory file. 0xFC and 0xFE indicate files pending deletion and bad blocks respectively, but they don't concern me in this setup.

To simplify, a block can be considered to be in use for a normal, healthy file, if the value is below 0xF8. For my purposes it is enough to check the presence of 0xFD, 0xFF, 0xFF for empty blocks.

The record's location within the FAT indicates the Block number this entry is pointing to. So, the third successive record is "about" the Block #3.

Files larger than 1536 bytes use multiple Blocks, and are indicated by a similar file number and an increasing sequence count. If there are only small files on the disk, the sequence numbers will always be zero. For one giant file, there is only one repeated file number whereas the sequence number increases.

An example portion of the FAT might look like this:

...FDFFFF FDFFFF 001000 00200 00300 00301 00302 FDFFFF FDFFFF...

The above describes three entries between empty records (FDFFFF). The first two, 1 and 2, are one-block files. The third, 3, has three blocks, indicated by the same filename and an increasing sequence count. The directory file would have 3 named entries.

Although normal file writing places the Blocks after each other, nothing says the sequences have to be in any particular order. So, a routine that reads a file from the image has to go through the whole record, picking up all data in order of the sequence indicator.

Just to repeat myself, the FAT looks like this in an empty image:

Block/Entry#, 12-bit file#,  12-bit seq#

0: 0xF8, 0x00, 0x00 (Block #0 contains the FAT)
1: 0x00, 0x00, 0x00 file/seq points to block #1 - Directory
2: 0xFD, 0xFF, 0xFF file/seq points to block #2 - (Earmarked for directory?)
3: 0xFD, 0xFF, 0xFF file/seq points to block #3 - (Earmarked for directory?)
4: 0xFD, 0xFF, 0xFF file/seq points to block #4 - (Earmarked for directory?)
5: 0xFD, 0xFF, 0xFF file/seq points to block #5 - (Earmarked for directory?)
6: 0xFD, 0xFF, 0xFF file/seq points to block #6
7: 0xFD, 0xFF, 0xFF file/seq points to block #7...
...
479: 0xFD, 0xFF, 0xFF file/seq points to block #479

Interestingly, my real QL seems to save files starting from entry 6 onwards, whereas the QLtools utility seemed to add a file to the first available position whatever, which is 2.

Perhaps it's my fantasy, but records 2-5 might be seen as "earmarked" for directory, although I'm unsure if it would matter that much speed-wise or anything. (It might even slow down the disk, who knows)

Again, a directory reading routine can assume the first directory block to be 1, but theoretically the rest can be anywhere else. It might be better for a reading routine to take its cue from the FAT, looking for file #0 components.


IMG Directory Block

Looking at the first directory block, a row of 0x30 values covers the first directory entry (the entry is the directory itself). After that the directory entries are assigned to files added to the disk.

I'm told the file name and file name length information within the file's own header is useless, and the file names within the directory file are definite. This is the only place to access the true length of the file in bytes.
Note that it's pointless to have a 16-bit value for filename length, but some documents have it this way.

Given the first entry in the directory list is already in use, 23 files can be added before new directory blocks are needed. For example, if five blocks are reserved for directory, this would mean the directory takes 5 * 1536 bytes = 7680 bytes. The directory contains file information and each file takes 64 bytes. This would give 119 files on the DD disk before yet another block would need adding.


IMG Block and track order

My first diagram about the disk image beginning is misleading, as it looks like the blocks and sectors are interleaved in a clear way up until the end of the disk.

Although int(block/6) gives the Track# you can expect to have the Block in, the Blocks are not ordered, and neither are the three sectors (512 bytes) inside a Block in a sequential order.

Ok, so let's look at this thing. Again, the image file is arranged in 480 * 3 = 1440 sectors, including the FAT.

As my first diagram shows, if a row is 512 bytes, the FAT appears on rows 0, 3 and 6. Then the directory contents start at 9th row of the image, padded with sixty-four 0x30 byte-values. The next row containing directory data is at 12th row, and the third is at 15th row.

How to make sense of the arrangement? Here the soulsphere site comes to rescue, but it had to be read in a certain way before it made sense.

Let's find the directory. As the first directory block is the second entry in the FAT, it points Block #1, the ordering which falls within Track #0. (Which has blocks 0-5).

Track #0 contents are in the following order:

TRACK #0
Offset: Block, Sector
0: B0S0 (each of these are 512 bytes)
1: B2S0
2: B4S0
3: B0S1
4: B2S1
5: B4S1
6: B0S2
7: B2S2
8: B4S2
(side2)
9: B1S0 first
10: B3S0
11: B5S0
12: B1S1 second
13: B3S1
14: B5S1
15: B1S2 third
16: B3S2
17: B5S2

We're looking for Block #1, so I've highlighted the three sectors it is made of. Looking at the serial ordering of the table, the Block #1 sectors 0, 1 and 2 are at offsets 9, 12 and 15. Glancing at the 512-wide arranged diagram, it can be seen the directory contents are indeed at locations 9 * 512, 12 * 512 and 15 * 512.

Let's look at the first saved file on the disk. In this example, looking through the FAT, it is at entry #6, thus pointing at Block #6. (In my examples the files fit in one block).

The Block 6 can be deciphered by looking at track #1, where the ordering is different from track #0. The sectors 0, 1 and 2 are at offsets 23, 26 and 20 (counting from track #0 beginning), so they are a bit backwards compared to what was seen on track #0.

TRACK #1
Offset: Block, Sector
18: B8S1
19: B10S1
20: B6S2 third
21: B8S2
22: B10S2
23: B6S0 first
24: B8S0
25: B10S0
26:B6S1 second
(side2)
27: B9S1
28: B11S1
29: B7S2
30: B9S2
31: B11S2
32: B7S0
33: B9S0
34: B11S0
35: B7S1

So, the routine that constructs a multi-block file into a linear buffer has to go through the FAT in file sequence number order, collect and append the sector 0,1,2 data for each block in that order.


Building IMG Disk header and FAT

The 95-byte header is at the beginning of the image.

The directory length (0x22-0x25) is the 16-bit value at 0x22 - 0x23 multiplied by 512, and added with the 16-bit value at 0x24 - 0x25 multiplied by 64. (Each file entry takes 64 bytes in the directory, including the directory file). This can be recalculated by scanning the FAT for unique file numbers, including the directory.

Free sectors at (0x14-0x15) can be recalculated by counting which of the 480 blocks are unused, and multiplying that by 3.

Below has the whole header described. I highlighted with green the locations that need changing in the header, when adding or removing blocks to the FAT. It may be a good idea to increment the update counter and randomize the values at 0x0D and 0x0E, otherwise the QL might think in some situations that the disk has not been altered and refuses to display an updated directory.

Disk ID and filename (0x00-0x0D)

0x00: 0x51 'Q' Header
0x01: 0x4c 'L' Header
0x02: 0x35 '5' Header
0x03: 0x41 'A' Header
0x04: 0x51 'Q' Label
0x05: 0x4c 'L' Label
0x06: 0x5f '_' Label
0x07: 0x44 'D' Label
0x08: 0x44 'D' Label
0x09: 0x20 ' ' Label
0x0A: 0x20 ' ' Label
0x0B: 0x20 ' ' Label
0x0C: 0x20 ' ' Label
0x0D: 0x20 ' ' Label

Disk information

0x0E: 0x92: Random value
0x0F: 0x53: Random value
0x10: 0x00: HI Update counter
0x11: 0x00: .. Update counter
0x12: 0x00: .. Update counter
0x13: 0x02: LO Update counter
0x14: 0x00: HI Free Sectors
0x15: 0x00: LO Free Sectors
0x16: 0x05: HI Good Sectors
0x17: 0xA0: LO Good Sectors = 1440
0x18: 0x05: HI Total Sectors
0x19: 0xA0: LO Total Sectors = 1440
0x1A: 0x00: HI Sectors per track
0x1B: 0x09: LO Sectors per track = 9
0x1C: 0x00: HI Sectors per cylinder
0x1D: 0x12: LO Sectors per cylinder = 18 (Double-sided)
0x1E: 0x00: Number of cylinders
0x1F: 0x50: Number of cylinders = 80
0x20: 0x00: HI Allocation block
0x21: 0x03: LO Allocation block = sectors / block = 3
0x22: 0x00: HI Directory Length: sectors (512)
0x23: 0x01: LO Directory Length: sectors (512)
0x24: 0x01: HI Directory Length: units of 64
0x25: 0x00: LO Directory Length:  units of 64
0x26: 0x00: HI Sector offset / track
0x27: 0x05: LO Sector offset / track

Logical-to-physical sector mapping table (18 bytes)

0x28: 0x00
0x29: 0x03
0x2A: 0x06
0x2B: 0x80
0x2C: 0x83
0x2D: 0x86
0x2E: 0x01
0x2F: 0x04
0x30: 0x07
0x31: 0x81
0x32: 0x84
0x33: 0x87
0x34: 0x02
0x35: 0x05
0x36: 0x08
0x37: 0x82
0x38: 0x85
0x39: 0x88

Physical-to-Logical sector mapping table (18 bytes)

0x3A: 0x00
0x3B: 0x06
0x3C: 0x0C
0x3D: 0x01
0x3E: 0x07
0x3F: 0x0D
0x40: 0x02
0x41: 0x08
0x42: 0x0E
0x43: 0x03
0x44: 0x09
0x45: 0x0F
0x46: 0x04
0x47: 0x0A
0x48: 0x10
0x49: 0x05
0x50: 0x0B
0x51: 0x11

A bunch of 0xFFs

0x52: 0xFF
0x53: 0xFF
0x54: 0xFF
0x55: 0xFF
0x56: 0xFF
0x57: 0xFF
0x58: 0xFF
0x59: 0xFF
0x5A: 0xFF
0x5B: 0xFF
0x5C: 0xFF
0x5D: 0xFF
0x5E: 0xFF
0x5F: 0xFF

The above was the header, and what follows is the FAT. I'll just give the start of the FAT - which is spread across the three sectors of block #0 (480*3 bytes)

0x60: 0xf8 Block #0, Points to FAT itself
0x61: 0x00
0x62: 0x00
0x61: 0x00 Block #1, Directory
0x62: 0x00
0x63: 0x00
0x64: 0xFD Block #2, Free
0x64: 0xFF
0x65: 0xFF
0x66: 0xFD Block #3, Free
0x67: 0xFF
0x68: 0xFF

... the FDFFFF repeated to the end of the first block.

After the first block the img can be filled with zeroes, and there you have it, an empty, formatted disk image in IMG format.


Reading from the IMG

It would be handy to find a file from the IMG based on its name, and then store it in pc memory or disk.

First, decipher the directory, because it's the way to look at file names. In this disk image it is record #1, but can use more records than the 1, so look for any FAT entries that have the file number 0 and then append the block (sector contents in sequential order).

File blocks, including directory blocks, may not always follow each other.

Now that the directory is accessible in a linear way, it can be read:

entrybase = start + 64 * file_no

offset:
0-3: Length of file
14-15: Length of file name
16-51: File name
52-55: Modification date (I don't use this)

(Just as in that diagram way up.)

The directory is looped through with file_no * 64. When the filename corresponds with the desired name, the needed file_no has been acquired.

Then the FAT is again looked through for blocks that have that file_no, and all sequential blocks belonging to file are appended in order to create a linear file, just like with the directory. The directory is a file just like any other so one can proceed by loading the file# 0 to a linear file buffer, then use that to load the desired file to another linear buffer.

From that linear file, file_length-64 bytes are then be stored to pc disk, beginning from the 64th byte, in case the 64-byte header is not needed.

When loading any file, directory or otherwise, as the blocks are found, the true sector offset within the image needs to be deciphered.

This row is taken from a table/algorithm that "converts" them into the funny order they are in the disk and provides a simple table from which to fetch the order. Instead of explaining this I'll just show the Processing routine I use to generate the ordered translation tables:

int []g_transblok=new int[1440];
int []g_transtrak=new int[1440];

void track_offset_generator() {
  int tphase[]={0,0,0,1,1,1,2,2,2,0,0,0,1,1,1,2,2,2,0,0,0,1,1,1,2,2,2};
  int bphase[]={0,2,4,0,2,4,0,2,4,0,2,4,0,2,4,0,2,4,0,2,4,0,2,4,0,2,4};
  int ptr=9;
  int count=0;

  for(int trk=0;trk<80;trk++){
    for(int sec=0;sec<9;sec++){
      g_transblok[count]=(bphase[ptr]+trk*6);
      g_transtrak[count]=(tphase[ptr]);
      g_transblok[count+9]=(bphase[ptr]+1+trk*6);
      g_transtrak[count+9]=(tphase[ptr]);      
      ptr++;
      count++;
    }
    
    count=count+9;
    ptr=ptr+4;
    if(ptr>17)ptr=ptr-18;    
  }  
}

If I want to know what is the offset row in the disk image of block x and sector y, I'll loop through the table with step of 2, until I have the pair. The result is that table position/2. Multiply by 512 and there's the address from the beginning of the IMG file.

int findoffset(int block,int sector) { 
  for(int i=0;i<1440;i++){
    if(g_transblok[i]==block&&g_transtrak[i]==sector)return i;
  }
  return 0;
}

If I want to know what is the block and sector at the disk image row n, I just grab the two values from row*2 and row*2+1.


Writing to the IMG

The end game of this Tour de Force, adding a file to the image:

Checklist:

Find the highest file number in the FAT. Your new file has file_no of that +1
Are there enough empty blocks in the FAT for the file? If No, go to ERROR

Is a new directory block needed? If No, go to APPEND

Are there enough empty slots in the FAT for the file + the new directory block? If No, go to ERROR

Extend the directory list file by creating a new FAT entry for file #0 in an empty slot

Give it a sequence number+1 compared to the highest existing block with file #0

APPEND:

Append the file data to a 64 bytes header, giving the complete file data and the length of the file in bytes

Write the file name and file length to the directory slot file_name*64

For every 1536-byte file block:

  • Write the file number and sequence number (starting from 0) to an empty FAT slot, the block# is the number of the first empty slot.
  • Write the three sectors of data to the image, offset row *512, the row is based on the assigned block# and sectors (0-2) values, through a conversion table.
  • Increase sequence number, for subsequent blocks (if any)

Refresh the value for number of free sectors in the header
Refresh the values for the directory list size in the header
Randomize disk Random # value
Increment the disk update counter

EXIT

ERROR:

Can't add the file! [END]


Into the HFE

Although the IMGs can already be converted to HFE with the HxC floppy emulator tools, as I had gone this far I wanted to get the IMG converted to HFE by my own means. This isn't as comprehensive, as I'm working with an already-formatted and HxC-converted file rather than creating a HFE totally from scratch.

The HFE file image for the QLDD is 2008064 bytes, which already clearly shows it is not a byte representation of the IMG.

Also, it has a 1024 byte file header as byte/word values, but the rest is bit-level information encoded in MFM (Modified Frequency Modulation) format suitable for the HxC to communicate with the disk controllers. MFM, the "Double Density", is a way to compactly encode the data and the clock signal into a waveform without wasting too much space/time.

Luckily the waveform generation stuff is not needed here, it suffices to know that information/data bits are interleaved with an encoding signal. I found it useful to have routines that access the HFE track bits directly.

Still, it is not entirely straightforward to transfer the IMG to an HFE, but it is not that difficult after understanding a few things. It also makes sense to approach the HFE only after doing the analysis on raw byte images discussed above. Also, writing data on an existing HFE ("formatted") is a bit simpler than building one from scratch.

Again, visualizing the HFE tracks in Processing was very helpful:



Before the sector bit data, the HFE file has a byte-readable HxC header. The most important information here are the byte offsets to each of the 80 tracks starting positions inside the HFE file:

base address: 1024 + track * 4
offset:
0: track offset LO
1: track offset HI (* 512)
2: track length LO (bytes)
3: track length HI (bytes)

(* 512 = multiply the 16-bit track offset value with 512 to get the offset from the beginning of the HFE file)

From the start of each track offset, reading every second bit after the first one gives the data bits for the track. Every bit in-between relates to the encoding. When reading the data, these are not really needed. Writing the data, these have to be coded to correspond with the data bits.

One header-related structuring still remains: within the sector data, disk side information is interleaved so that the first 256 bytes relate to side 0, the next 256 bytes to side 1 and so on. To make things simpler I make the whole HFE data linear before working on the image, then de-linearize it before saving it.

Now, supposing we've found the start of the disk data (more on that below) that corresponds with the header start in the raw byte image:

Q        L        5        A        Q        L        _        D        D
01010001.01001100.00110101.01000001.01010001.01001100.01011111.01000100.01000100.
00000110.00010001.11000000.00011110.00000110.00010001.10000000.00011001.10011001.

The bits have been broken into the data bits (above) and the encoding bits below. The data bits are obviously good ol' ASCII here, and the encoding scheme is not too hard to see:

-Start with a zero encoding bit.
-Each time there is a zero in the corresponding data bit, and the previous data bit is zero too, switch the encoding bit to one
-Keep the value 'one' until the data bit is one, at which position change the encoding bit to zero

To read a byte offset from the beginning of the track address, the address is offset*2. Then take 8 bits, hopping over every second bit, and build the byte that way.

Again, reading the encoding bits is in no way needed for reading the HFE image. Writing the encoding is necessary when converting an IMG to the HFE.

Although I'm working on the same image all the time, let's remember the start point of the disk is not in any absolute location inside the track #0 data. It is better to seek the sector identifiers and the data blocks through the ID markers spread across the disk.

The information about disk data is in areas indicated by A1A1A1. The A1A1A1FB appears after A1A1A1FE, which is the track identifier.

SYNC byte, before Index Address Mark or ID Address Mark
C2,C2,C2,FC/FE
14,14,14,01

SYNC byte, before Data Address Mark
A1,A1,A1,FB
0A,0A,0A,00

The corresponding sync byte clock signals are not 'normal', and help make detecting the marks less ambiguous. This Atari ST site cleared a lot of things for me.

Inside one track, these can be found:

A1,A1,A1,FE, trk#,00,01,02,CRC,CRC
A1,A1,A1,FE, trk#,01,01,02,CRC,CRC
A1,A1,A1,FE, trk#,00,02,02,CRC,CRC

A1,A1,A1,FE, trk#,01,02,02,CRC,CRC
A1,A1,A1,FE, trk#,00,03,02,CRC,CRC
A1,A1,A1,FE, trk#,01,03,02,CRC,CRC

A1,A1,A1,FE, trk#,00,04,02,CRC,CRC
A1,A1,A1,FE, trk#,01,04,02,CRC,CRC
A1,A1,A1,FE, trk#,00,05,02,CRC,CRC

A1,A1,A1,FE, trk#,01,05,02,CRC,CRC
A1,A1,A1,FE, trk#,00,06,02,CRC,CRC
A1,A1,A1,FE, trk#,01,06,02,CRC,CRC

A1,A1,A1,FE, trk#,00,07,02,CRC,CRC
A1,A1,A1,FE, trk#,01,07,02,CRC,CRC
A1,A1,A1,FE, trk#,00,08,02,CRC,CRC

A1,A1,A1,FE, trk#,01,08,02,CRC,CRC
A1,A1,A1,FE, trk#,00,09,02,CRC,CRC
A1,A1,A1,FE, trk#,01,09,02,CRC,CRC

Look familiar? They are the sector IDs for the six Blocks within a Track.
Each sector ID is soon followed by a Data ID, which is then followed with 512 bytes.

A1,A1,A1,FB,[512 bytes of data],CRC,CRC

But now we also have something that wasn't present in the raw byte disk image at all: CRC values.


CRC

So, the HFE format project led me to the world of calculating Cyclical Redundancy Check values, which I found to be a huge topic in itself and won't discuss it too much here.

This site opened up to me the "easy" way to calculate the results and also warned about erroneous calculations. One problem with CRC is that an incomplete implementation may result in correct CRC in some cases, something which led me astray for a while.

This on-line calculator was helpful for comparing my results with different types CRC initial values.

The QL disk track ID I was looking at has the values A1A1A1FE00000102, which is the message for which I need the CRC, as the 16-bit CRC is stored after it. In this case the CRC was CA6F.

I toiled away with another Processing sketch and visualization to get results and a sufficient understanding on the topic. Although CRC is potentially very math-heavy, the process can actually be made simple with just throwing bits around and visualizing the thing until it gets right.

For the QL Floppy CRC I need to use initial value of 0x84CF to get what would be equal to results of "CRC-CCITT 0xFFFF direct":

The message is preceded with the initial value. After the message, as an augmentation, 16 zero-bits are added.

The above image is off but shows the principle.

The bits are churned by XORing the polynomial 0x1021 (in practice 0x11021) with the bits "above", bringing in an extra bit from the message/augmentation as it proceeds from "left" to "right". After the limit is hit at the "right edge", the value we are at is then the CRC.

Writing to an existing formatted HFE disk, the sector ID CRCs need not be overwritten, as their contents do not change. Should this be needed, then CRC the 8-bytes starting from the first A1, and then write the 16-bit result after the data.

With the 512-byte file blocks, the 516 bytes starting from the first A1 are CRC'd, then write the 16-bit value after the data. That's a helluva lot to CRC, and a poorly optimized routine (*cough*) can result in a fair amount of churning.

Whenever data changes, the encoding signals have to be regenerated. This is best done for the data areas, starting after the A1A1A1FB, with the initial bit value as 0. The encoding bits are written with the above-mentioned rules, up until and after the data block and the byte after the CRC value. (If you rewrite sector IDs and for some reason they change, the encoding bits have to be rewritten there too)

When writing the encoding bits, the whole track encoding could be calculated from beginning to an end. But when working on an existing formatted HFE image, I felt it was easier not to encode areas that don't need re-encoding, as then the anomalous sync signals within the ID portions do not need to be addressed.

Maybe I'll publish it, one day.


Monday, 30 April 2018

Adding weights to chess pieces


A proper Retrogaming mod: I added weight to my cheap chess set.

This required a lot of boring stuff with a power drill and a crap selection of drill bits.

First I removed the felt, evened the surface a bit and punched entry point markers using a drill tip. Already removing the unevenness and the poor felt the pieces kept more firmly upright.

I then drilled all pilot holes with a thin 2-3mm drill. It would be good to get these straight, but as I did not want to build a setup for this I had to accept some deviation.

Then, 6mm holes. This was maybe the easiest part of the whole drilling process, as the 6mm drill cuts the material pretty easily. The drill needs a tape marker or a limiting piece to avoid drilling too far. I did think I could "feel" my way but after a couple of mistakes I had to attach the tape. A very simple measure that avoids a lot of hassle.

Not to scale
I drilled all 10mm holes using a two-piece jig to hold the piece more firmly, then inserted the screw.

The simple jig only half-does the job, I had to secure the pieces with my other hand. Luckily the piece bottoms are all the same size, not really how it ought to be, but at least this way they all fit the same jig.

The 10mm drilling was quite tough and kludgy to do, I found it useful to switch between reverse and forward gear while drilling.

It would have been much better to drill the 10mm holes before the 6mm ones to preserve the function of the pilot holes. But the 10mm is already a bit tricky with a power drill and the drill head is not the best quality. So I reasoned the 6mm holes could be done first to remove material and make it a bit easier for the 10mm.

The jig hole is covered with furniture pads, so I did not really need to unclamp the thing.
I used longer screws for the Kings and Queens so they have a teeny bit more weight to them than the other pieces.

The light bolts do not radically add weight, but it's enough to give a bit of a feel to them. For a while I thought about using some kind of lead pellets, but the screws have less work stages to consider and I already had enough of them. Even if the machine screws do not properly attach to the wood they still stay firmly enough on their own.

I also made some changes to the appearance of the pieces, I'll get back at that when the set is more finished.



Sunday, 15 April 2018

AKAI MidiMIX & Processing Midibus



I got this AKAI MIDImix USB controller. It's principally meant for Ableton, like apparently half the stuff nowadays is. My only real interest is to connect it to my computer with the Processing MidiBus library.

Previously I have have been messing with MicroKORGs, Electribe and a Nanokontrol 2, so a controller like this is not a big deal. Compared to synths, the major difference is that the lights can be set independently of the key presses. They even need to be set because the physical button presses only send information about the key press itself and do not change the light state.

I'll look at the Processing midi library a bit and then at the AKAI mapping.


Midibus

MidiBus is a clever library for Processing that eases the use of MIDI considerably. It handles input, output, program change, control change, raw midi data on multiple channels. Here I'm not going into song player routines, which are a bit tricky but doable. (The Midibus does not have a player).

After installing MidiBus to the Processing install, this will activate it in the source:

import themidibus.*;

void setup()
{
  MidiBus.list();
  mybus=new MidiBus(this,1,2);
  println(mybus);
}

The "(this,1,2)" can be adjusted to input and output of the given list of MIDI devices you get from running the sketch.

The MidiBus.list() produces a list to the console with the relevant information. My USB/MIDI interface is an UM1G, and as the AKAI MIDImix works through a separate USB cable (not a MIDI cable) it will be listed as a separate device:

Available MIDI Devices:
----------Input----------
[0] "UM1G [hw:2,0,0]"
[1] "Mix [hw:3,0,0]"
[2] "Real Time Sequencer"
----------Output----------
[0] "Gervill"
[1] "UM1G [hw:2,0,0]"
[2] "Mix [hw:3,0,0]"
[3] "Real Time Sequencer"


After the Midibus has been set correctly, sending a note is simple:

mybus.sendNoteOn(midi_channel, pitch, velocity);

You need to shut the notes too:

mybus.sendNoteOff(midi_channel, pitch, velocity);

For example, these could be directly inserted within the Processing keyPressed() and keyReleased()events to facilitate a simple MIDI keyboard.

Control Changes for things like Modulation Wheel, Cutoff, Resonance, can be sent like this:

output.sendControllerChange(midi_channel, CC, value);

Just as an aside, some devices use NRPN (Non-Registered Parameter Number) for similar things. In such cases the data has to be sent as raw MIDI information. This worked for my Korg Electribe:

output.sendMessage(0xb0, 0x63, 0x05);
output.sendMessage(0xb0, 0x62, NRPN);
output.sendMessage(0xb0, 0x06, VALUE);  

The 0xb0 signifies MIDI channel 1, whereas 0xb1 would be 2, and so on. Good manufacturer manuals and datasheets should give Control Change and NRPN values for each device.

But it's also possible to find out the CCs and note values by displaying the incoming MIDI data. The values can shown by adding controllerChange() and noteOn() events to the sketch:

void noteOn(int channel, int number, int value){
  println("Received note channel:" + channel);
  println("Received note number:" + number);
  println("Received note value:" + value);
}

void controllerChange(int channel, int number, int value) {
  println("Received CC channel:" + channel);
  println("Received CC number:" + number);
  println("Received CC value:" + value);
}  


AKAI midimix

Ok, back to the MIDImix. Physically, the knobs could have been taller, it's not easy to get a good grip of them. Also the faders could have been a bit longer but this is less of a problem. Yet the MIDImix is compact and quite cheap, although not as cheap as a Nanokontrol might be.

The nice thing with these modern controllers is that the data is transmitted through an USB cable and it's also USB-powered, so no power cable / MIDI cable hassle.


AKAI works on MIDI channel #1, and I didn't see how it could be changed. For the purposes of my tests I simply disconnected the MIDI cable from my UM1G box.

All potentiometer and slider values are Control Change messages with the value 0-127 with them.
The buttons are "keys" that send a velocity of 127 with them.

Potentiometer CCs:

I   II III  IV   V  VI VII VIII
16  20  24  28  46  50  54  58        TOP ROW
17  21  25  29  47  51  55  59        MIDDLE ROW
18  22  26  30  48  52  56  60        BOTTOM ROW

Slider CCs:

19  23  27  31  49  53  57  61  (62 = MASTER)

MUTE button Note values:

01  04  07  10  13  16  19  22  (27 = SOLO)

Individual SOLO button Note values: *)

02  04  08  11  14  17  20  23

REC ARM button Note values:

03  06  09  12  15  18  21  24

Bank Left Note value:

25

Bank Right Note value:

26

Solo button Note value:

27

*) Holding down the "SOLO" key shows the individual SOLO status for each channel instead of the MUTEs. So, sending a SOLO light change only shows up if the "SOLO" key is held. In other MIDImix devices there may even be separate lights for the SOLOs. Pressing the "SOLO" key does send a note value, though.

Sending note-ons to MUTE, REC ARM or Bank button values with velocity 127 will turn the associated lights on. Sending note-ons with velocity 0 will turn the lights off. Sending note-offs does nothing. The "SOLO" key cannot be lit this way, possibly there's no LED at all.

Pressing SEND ALL will cause the MIDImix to send all the above values in succession. This key also may not have a LED in it.


Tuesday, 3 April 2018

QL Tetroid Gold Card clone


Last year, I was so eager to get Tetroid's Disk Interface for the QL I didn't notice Tetroid's Miracle Gold Card clone - that would have been more interesting for me. I cursed myself a bit, waited for some time, then decided to order it anyway.

The Gold Card has a 68000 processor, a floppy disc interface, battery for backing up the clock/date and 2 megabytes of memory. I had a slow moment, thinking, "How can this be, the QL cannot address more than 1MB?" But of course the 68000 can access more memory than the 68008, that's the whole point.

The speed increase (~4x they say) is the main reason I was interested in the Gold Card. The speed is not about processor frequency, it's just that the 68000 does not need to "convert" 16/32 bit instructions to a 8-bit data path the way the 68008 does.

Theoretically, if a program is coded in a way that avoids 68008 bottlenecks, it might not be radically faster on the 68000.

The even faster Super Gold Card is probably eventually going to be cloned too (I'm hoping), but I just could not wait.

Unlike the Tetroid Disk Interface (TDI), there is no Compact Flash card reader built in. The Gold Card has a floppy disk interface which together with my HxC Floppy Emulator is enough for loading programs from an SD card. But the card seems to work well with the QL-SD and Minerva ROM, so I can have both FLP and SDC connected.

To make the difference between the three products clearer, I've created this table:

                      TDI      GC       QL-SD
Card Reader:          YES(CF)  NO       YES(SD)
68000/Acceleration:   NO       YES      NO
Memory:               800K     2MB      NO *)
Disk interface:       YES      YES      NO
Battery-backed clock: NO       YES      NO
Toolkit 2:            YES      YES      NO
Minerva ROM:          NO       NO       YES

*) The file allocation table takes memory

The QL-SD is not really comparable, but I included it to make clear how these devices might complement each other. Note also that none of the solutions enable a direct file-transfer between a PC/MAC file system and the card reader.

Looking at the Gold Card board, it is the same professional quality as the TDI. Installing was easy. There are no jumpers or settings to consider, nothing to screw or un-screw, just remove the extension port cover and push the card in gently along the rails.



The card is a bit longer than the TDI, although it does not extend over the computer footprint.

When connected, take care: Do not grab the card when picking up the computer, as this can bend the boards inside.


Booting & BASIC test drive

Although the booting sequence must be technically faster, there is also more memory to check and therefore it takes a tiny while. Then I get the "Gold Card v 2.49" message and I'm off to superBASIC.

With the Minerva ROM, I have to look at the added boot option screens - the QL-SD also needs a short waiting time. For some reason I had to do my mode selection "twice" if I go for the F4/Dual screen. But the Minerva boot and memory check have been optimized so it's pretty quick.

It's instantly clear the speed makes it much more pleasurable to work in the BASIC environment. I no longer have a feeling the system stays behind my moves. Before the upgrade, all listing, editing, typing, scrolling and similar actions were about bearable, now they are quite slick.

Again, TK2_EXT activates the Toolkit 2 extensions (the command itself is faster) and after that you can use the semi-fullscreen editor ED, FSERVE file server, or use WTV to set the BASIC window size to work better with a television.

Just eyeballing a couple of BASIC graphics routines shows the 4 x speed increase is quite real. Testing also highlights how slow the SuperBASIC is to begin with, so I'll have to say even now the BASIC does not give lightning-fast graphics.

When comparing an unexpanded QL with a Minerva ROM+68000, not all speed increases are due to the processor, as Minerva improves the ROM routines a bit. Taken together, SuperBASIC on Minerva+68000 is clearly 4 times faster and even more.

For example, the following line-drawing test could finish nearly 7 of the 8 color sweeps before the unexpanded QL without Minerva could finish even one.

100 MODE 8
110 PAPER 0: CLS
120 FOR I=0 TO 7
130 INK I
140 FOR Y=0 TO 100
150 LINE 0,0 TO 100,Y
160 NEXT Y
170 NEXT I

With output something like this.
QED text editor is smoother and begins to remind me of typing on ST/Amiga editors. My QMAC compiler setup works faster, and I guess the file access from RAM drive should also benefit from the acceleration.

The Qliberator-compiled BASIC increases the speed further. The compiler by itself does not do that much and my rough estimate is that even the non-compiled BASIC on 68000+Minerva is faster than compiled programs were on a completely unexpanded QL.

There are probably accurate benchmarks and testing software around, but this kind of simple observations are enough for me.


Final remarks

Decades after, it's easy to say that a full 68000 is what the QL should have had in the beginning. But old 68000 processors were costly and they also took up room on the board (64 vs 48 pins). QL would have jumped to a different price category and likely eaten up more desk space. The 68008 helped to make a very cheap and tiny computer with enough 16-bit sensibilities, yet more portable than many "portables" were back in the day.

But what about the Miracle Card Clone in this day and age? Because the card has a processor, memory and a disk interface, maybe Tetroid might clone a full computer, integrate the (Super) Gold Card and fit it all inside original QL case and connectors? That would be wonderful.