Rockbox

Tasklist

FS#9787 - iPod 5G TV out

Attached to Project: Rockbox
Opened by Boris Gjenero (dreamlayers) - Monday, 12 January 2009, 07:05 GMT
Task Type Patches
Category Drivers
Status Unconfirmed
Assigned To No-one
Operating System iPod 5G
Severity Low
Priority Normal
Reported Version Daily build (which?)
Due in Version Undecided
Due Date Undecided
Percent Complete 0%
Votes 0
Private No

Details

Here is a preliminary patch for TV out on a 5th generation iPod. Play a BMP file and TV out will activate. NTSC is the default. If the file has 576 lines, the output will be in PAL. The image stays until another is loaded or TV output is disabled via the debug menu..

The following formats ought to work:
720x480, 24 bpp for NTSC
720x576, 24 bpp for PAL
Many programs including Windows Paint and ImageMagick convert can write such files. Other resolutions and bit depths may work. For example WPS background screens display properly but other WPS bitmaps don't.

Known issues:
- More work needs to be done to integrate this with Rockbox. No, you cannot view the Rockbox interface, JPEGs or mpegplayer output yet.
- When the LCD is updated, there is a temporary disturbance on TV output
- http://en.wikipedia.org/wiki/Macrovision#Analog_copy_prevention is enabled. Yes, the hardware can disable it, but I haven't figured out how to do that yet.
- The original firmware probably outputs images differently. It can load programs into the BCM which perform decoding there, and that is probably faster.
This task depends upon

Comment by Boris Gjenero (dreamlayers) - Monday, 12 January 2009, 07:18 GMT
One important limitation I forgot to mention: When loading a BMP, audio playback is stopped because the file is loaded into the buffer.
Comment by Björn Stenberg (zagor) - Monday, 12 January 2009, 07:49 GMT
Nice work!
Comment by Boris Gjenero (dreamlayers) - Saturday, 17 January 2009, 19:17 GMT
Thanks!

I was just trying to remove the interference caused by LCD updates. Changing the lcd_write_data loop to slow it down and check for write ready doesn't change the interference, so I guess it happens when the BCM executes the LCD update command, not when Rockbox writes to the BCM. One solution would be to execute the command during the TV out vertical retrace. This would require knowing when the retrace starts.
Comment by Boris Gjenero (dreamlayers) - Sunday, 18 January 2009, 17:55 GMT
Here's an updated patch which disables Macrovision using BCM command 14. While playing with this I noted:
- Executing command 14 once seems enough
- Enabling TV out before sending command 2 or 3 (to display BMP data) seems unnecessary
- There's a significant signal dropout while uploading new BMP data, which means this method cannot be used to display video
- 16 bpp BMP files aren't displayed properly.
- BMP files with reverse row order crash the BCM.
Comment by Boris Gjenero (dreamlayers) - Friday, 23 January 2009, 06:55 GMT
Here are some benchmarks because someone asked and I was curious. I guess the 320x240 benchmark is fast enough for video. However, because the TV out signal is interrupted when changing images, it cannot be used for video.

30 Mhz:
320x240, NTSC: 52.5 FPS, 42% CPU
720x480, NTSC: 6.6 FPS, 24% CPU
720x576, PAL: 5.5 FPS, 24% CPU

80 Mhz:
320x240, NTSC: 72 FPS, 21% CPU
720x480, NTSC: 7.7 FPS, 11% CPU
720x576, PAL: 6.4 FPS, 10% CPU

The speed increase at 80 Mhz comes purely from faster loading of data into the BCM. The amount of time the BCM takes after that is unchanged.
Comment by Sadurní (sadur) - Tuesday, 27 January 2009, 18:28 GMT
Nice work. This is an start to a feature I would like to have: plug my iPod into the TV and view (at least) the wps screen on it. It would be great!
Thanks.
Comment by Boris Gjenero (dreamlayers) - Wednesday, 28 January 2009, 16:34 GMT
Currently the TV signal is interrupted while changing the displayed image. This is unsuitable for rapidly changing images such as video or WPS. Currently I'm only planning to add TV out support to the JPEG viewer.
Comment by Boris Gjenero (dreamlayers) - Monday, 02 February 2009, 05:41 GMT
I just noticed a problem: The current method used to disable TV out (accessible via the debug menu) only decreases power consumption a bit and does not return it to its normal level (without TV out). I either need to figure out how to do that or write code to reset and initialize the BCM.
Comment by Boris Gjenero (dreamlayers) - Friday, 06 February 2009, 07:34 GMT
I tried to dump BCM RAM via the bcm_read32 function, hoping to find the actual frame buffer. It seems the 1.25 MB of internal SRAM appears at 0-0x13FFFF and can be dumped. After that there's garbage that changes, and the address space wraps at 2 MB (0x200000). This means the 4 MB of SDRAM and other parts of the BCM are not accessible. There may be a way to read some of that because "Bypass mode allows host access to memory-mapped peripherals with BCM2722 in power-down."

In the SRAM, I found a copy of the LCD data at 0x238E0, but altering that does not alter the LCD. Other than that, there's the firmware, some data, and a lot of what looks like uninitialized memory.

The 4 MB of SDRAM is at 0xC0000000, and while it can't be read now, it can be written. The TV out bitmap is written ot 0xC0200000, but that's not what gets displayed. The actual data being displayed is somewhere in 0xC0000000-0xC0200000, and altering it does alter TV output. Writing all over the SDRAM does not alter the LCD.

So, once the actual location and format of the TV out frame buffer is determined, what's displayed via TV out may be changed just by writing to BCM SDRAM, without having to execute a BCM command which interrupts TV out. This may allow showing the WPS and playing video.
Comment by Sadurní (sadur) - Friday, 06 February 2009, 15:00 GMT
Great news! :)
Comment by Boris Gjenero (dreamlayers) - Saturday, 07 February 2009, 19:26 GMT
The frame buffer is at 0xC0000000 in the BCM.
Format is 24 bit BGR (ie. it starts out with: blue[0] | (green[0] << 8) | (red[0] << 16) | (blue[1] << 24)).
Pixels are ordered in the usual frame buffer way (ie. pixels left to right, scanlines top to bottom as displayed in a frame (fields not separate))
Luminance must be at TV levels (ie. 16-235 not 0-255).
Due to overscan, the edges of the displayed area are not visible.

The hardware is capable of multiple video modes. If the command enabling NTSC TV out is given a 320x240 bitmap, hardware will be in 320x240 mode. If it was given a 720x480 bitmap it will be in 720x480 mode.
Comment by Boris Gjenero (dreamlayers) - Sunday, 08 February 2009, 05:16 GMT
I've written code to translate and copy lcd_framebuffer to the TV output framebuffer. There is no interference when uploading data to SDRAM, the speed is decent and it ought to be usable for video. I was using the Rockbox interface and various plugins. Doom was playable. The only thing I can't do yet is watch video; for that I need a modified lcd_write_yuv420_lines function which writes in TV output framebuffer format.

Writes to SDRAM seem to have a bit of a DMA cache coherency problem. At the end of an update, a bit less than the size of one line of text at 320x240 is mostly not updated. Everything up to that point is updated, and the data is not lost, just delayed until more data flushes it out of the cache. I don't know how to flush the BCM's cache, but I expect the worst case workaround is to just write that much extra data, which isn't too bad.

Regarding use of lcd_framebuffer: it's just the easy way. It's a bit slower and it limits colour resolution (RGB is 565 instead of 888). The other way would be to write 24 bpp LCD code. This would mean writing a new driver and adding new code to any plugins you want on TV out. I think it makes sense to use lcd_framebuffer to give TV out support by default and only add 24 bpp support for things which benefit from 24 bpp (eg. image viewers) or additional speed. Right now my translation code uses 64+32 bytes of IRAM for lookup tables. I hope that's okay. I think it's faster than intelligent use of shifts but I suppose it should be tested.

Any thoughts on what to do about overscan (edges of the screen getting cut off)?

Comment by Boris Gjenero (dreamlayers) - Monday, 09 February 2009, 21:54 GMT
Here's the code I mentioned in my previous comment. There's still no user interface or mpegplayer support. A patched copy of Rockbox will put the LCD to sleep and instead output the LCD image via TV out in NTSC mode. To flush the BCM cache I'm writing some extra data at the end of updates. Writing 0x30 or 0x31 to BCM_CONTROL seems to flush the cache, but it also creates the same interference seen when executing LCD update commands.
Comment by Sadurní (sadur) - Tuesday, 10 February 2009, 07:05 GMT
Good job!
I'll try this patch as soon as possible. Maybe this afternoon.
Thanks a lot.

Comment by MichaelGiacomelli (saratoga) - Wednesday, 11 February 2009, 01:09 GMT
>Format is 24 bit BGR (ie. it starts out with: blue[0] | (green[0] << 8) | (red[0] << 16) | (blue[1] << 24)).

I don't suppose theres any clue about using YUV? I'd be surprised if the chip couldn't handle it, and it would make mpegplayer a good bit faster.




Comment by Sadurní (sadur) - Wednesday, 11 February 2009, 15:55 GMT
Well, well... well done!
I've tried it and works! I have no idea of what to do with overscan, really it's a pain.
With "There's still no user interface", did you mean that is not configurable, didn't you? Is there any chance to activate/deactivate at runtime?
Again, thanks a lot.
Comment by Boris Gjenero (dreamlayers) - Wednesday, 11 February 2009, 20:18 GMT
I understand the benefits of YUV. I don't know if it's possible to make the BCM TV out hardware or the cut-down BCM firmware in flash display YUV data. The BCM firmware in the resource partition definitely does support YUV (and a lot more). I would probably have to look at how the on-disk original firmware uses the BCM, and I've been avoiding that because I think it's probably much more complicated.

Regarding overscan, I guess I'll have an option to choose the resolution. At 320x240 you get overscan and at 720x480 or 720x576 you get a small centered image. I'll see if other resolutions are possible too.

Yeah, by "interface" I meant a means to turn it on and off and configure it. Actually, I don't know how to tell the BCM to totally turn off TV out. However, I do know how to turn off and reinitialize the BCM, and I've submitted that code as  FS#9890 . I am thinking of basing this patch on SVN+ FS#9890  and using BCM restart to turn off TV out. I think that would be cleaner than duplicating code in both patches.

Finally, here's a bit of information on the BCM firmware in the resource partition:
Look at your iPod firmware partition, or rename a .ipsw file to .zip and extract the firmware file. Search inside for "IPODRESOURCFAT16". What you see is a FAT16 boot sector, which starts 0x2B bytes earlier with 0xEB. Copy data starting with 0xEB into a new file and mount (eg. -o loop) or view (eg. UltraISO) that FAT16 filesystem.

In Resources/VideoCore/Boot you have a vmcs.bin which may be the firmware loaded into the BCM once the OF boots from disk. In Resources/VideoCore/Library you have vll files. Those are ELF DLLs which get loaded into the BCM. GNU objdump and nm can provide some info (eg. try "nm -D" or "objdump -x"). Unfortunately, details of the architecture and instruction encodings aren't known so you cannot disassemble those files. I guess the only development tools available are this version of Metaware:
http://www.broadcom.com/products/software/mobmm_metaware.php
http://www.arc.com/downloads/success_stories/C12260_Alphamosiac.pdf
Comment by MichaelGiacomelli (saratoga) - Thursday, 12 February 2009, 04:51 GMT
>Unfortunately, details of the architecture and instruction encodings aren't known so you cannot disassemble those files.

The pictures of the emulator in your first link look like they're based on MIPS to me.
Comment by Boris Gjenero (dreamlayers) - Sunday, 22 February 2009, 15:31 GMT
>The pictures of the emulator in your first link look like they're based on MIPS to me.

In the disassembly window, are mov or addcmpbge MIPS instructions? In any case, the instruction encoding is different. It seems MIPS uses fixed-length 32 bit instructions and the BCM uses something different. Data certainly isn't encrypted, and I don't think the code is either; it's just weird. Some of these Alphamosaic patents may provide additional information: http://www.google.com/patents?as_pasgnee=alphamosaic . Several show 48 and 80 bit instruction encodings.

The interaction between the on-disk portion of the PP and BCM firmwares is quite complex. I'm tempted to say an order of magnitude more complex. Figuring it out would allow neat things, such as using DMA to send data to the BCM, sending YUV data, decoding video on the BCM and using BCM-accelerated OpenGL ES. However because of the complexity, trying to figure it out might be kind of crazy.
Comment by Antoine Cellerier (dionoea) - Sunday, 08 March 2009, 20:30 GMT
Awesome patch! I've synced it to svn r20248.

I seem to be having some problems when using it. Pluging/unpluging the line out or headphone jack seems to trigger some freezes. I've only tested for like 2 minutes so I don't have anymore info.
The screen is also missing a few pixels on the left and top.
Last but not least, when the LCD shutdown (or at least backlight shutdown) is triggered, TV out is permanently disabled and I get the last rendered frame on the ipod's LCD. (which isn't updated anymore)
I'll try to investigate all of this a bit more as well as other resolutions.

Thanks once more to Boris for your work on the iPod Video port.

Edit: The screen freeze stuff seems to be due to the newly commited LCD_SLEEP stuff. If I disable that it's better (at least the TV video output always stays on).
Comment by Boris Gjenero (dreamlayers) - Sunday, 08 March 2009, 23:13 GMT
I've been busy with other things so I've neglected this patch.

I have not experienced any problems when plugging/unplugging headphones, line out or the dock connector. Rockbox shouldn't even know if line out is connected to anything.

The headphone connector jack is non-standard. There is an extra ring which is used for video out. Here's one image of the corresponding plug: http://www.colordrives.net/ipod%20video%20cable/ipod%20video%20cable-2.jpg . If TV out is on, I hear the TV out signal as a buzz while plugging in and unplugging headphones. When a standard headphone jack is plugged in, the ground ring extends over that extra ring and presumably shorts out that TV output, but TV output via the dock still works.

Regarding the missing pixels at the edges, that's due to overscan. See http://en.wikipedia.org/wiki/Overscan . The easiest solution is to use a higher resolution and put the LCD image in the middle. I don't know if there's a way to alter TV out image size and position.

Regarding LCD_SLEEP: the same chip which interfaces to the LCD also does TV out, and lcd_sleep() turns it off. Because of that, TV out needs to be reinitialized in lcd_awake().
Comment by Antoine Cellerier (dionoea) - Monday, 09 March 2009, 13:37 GMT
I use the iPod dock's s-video connector for the video output. I didn't know about the special jack stuff.

I'll cleanup the synced patch and fix the lcd_awake() problem.

I've also tried using some "standard" PAL resolution (or NTSC) and have a few issues doing so. I still need to play with it a bit, I'm not sure if the issue is due to font rendering only or to a flawed width setting (I changed both the LCD_{WIDTH,HEIGHT} defines and the bitmap header in the lcd driver) but horizontal lines or list selection gradients display ok, while all the text or even icons are unreadable. A nasty side effect of using such a big resolution (like 720x576) is that drawing to the BCM chip takes way too much time which causes playback to be completely broken (like 1 sec of audio every 5 seconds). That's with the default WPS (builtin, not Cabbie v2). I haven't had the opportunity to try with a WPS without any fast updating stuff like the peak meters.

The other thing I was wondering about was if it was possible to detect when the s-video cable is plugged in to the dock (or your special TV out headphone jack). If it is (could it be something like the resistance value changing on some of the pins like for some of the accessories?), I'll try implementing auto TV out when plugged in. That would limit TV out to the LCD resolution, but at least it'd be usable without having to have 2 rockbox firmware builds depending on where you want it to display.

Loading...