CPC web browser/Internet Applications

[rpalmer]

Hello everyone,

Well I have been very busy getting my CPC-Ethernet card up and running, but have yet to test it on a real connection.  I have an in-built test mode where I can insert messages and inspect out going ones and it appears to works as expected.  To date it can setup the TCP/IP stack for DHCP and ARP with simulated test data (as captured with Wireshark). Its next phase is with a simulated telnet session as an external application, which will take some time as i had to capture a session using wireshark on the PC and it seems like there is an enormous amount of data to simulate.  But if this does goes as expected then there is no reason why a real connection would not work (lets cross my fingers that is does work).

So now is the time for a web browser, albeit one that can only handle text (I can make handle some graphics, but given that they are often in the Mb's that is going to be one hell of a task!). The eventual first beta version will make it ready to demonstrate the apps for the TCP/IP stack with the API (so people can see a real example of the TCP/IP API in action).

My first phase is to simply get the code to compile with a minimum set of functions where one can load a simple HTML file and display on the screen.
There will be no internet connection (in the first beta) as the telnet testing needs to happen first whereby I can get it accept external messages and then send messages out on real connection.

My first impression of the compiled code appears it will fit with the lower 16K memory with some of the code sitting in the external banks, however the severely cut down version when updated will no doubt extend the current code size to a point where multiple compiles are required to get a working version.

The web-page content can only be 256K in size which appears to more than enough for now, but as the memory management is extended it maybe possible for the 4Mb expansion to be used for bigger pages.

The cursor controls are via the keyboard, but these can be configured to use the joystick and hence any mouse device (aka bryce productions). The cursor controls and hyperlink selection will operate as in existing browsers, so no special keys sequences are required.

Please understand that at this stage the code only compiles (and would not run as the application requires fonts, configuration and test data to be setup). All of which is also being created at this time.

The browser will only handle a sub-set of the HTML V4.0 specification as shown in design notes, but as the app develops there is likely to be more to come.

If anyone wishes to see the code as it currently stands, you find the attached file showing what I have done so far.  Included in the zip is some design notes.

The first useable version (when completed soon) will also include a .DSK image with all the required files to start up a session of the browser to view a simple HTML file.

[Gryzor]

Oh man, sfi-fi coming to a CPC near you...

[TFM]

Excellent work Richard! You invested a lot of time and work. This looks all very promising 

[rpalmer]

hello everyone,

Well I have managed to get an executable version of the browser.

The attached code has compilable executable, file1 and file2, but the HTML rom has some fixed addresses that need correcting (if the routines are to be accessed).

The functions I have got working are:

- Opening a URL (but does not load it)
- Links Manager to view and select, I have coded save and load (but not tested it)
- Information (Version/History)

To run the app within winape:

1. Install the ROMs (any valid sockets)
2. Initialise TCP/IP with - |TCPIP.INIT,1500,1,32768
3. Execute the app with - |TCPIP.APP,"cpcb",&170

The Winape machine is configured as 6128 classic with all 512K expansion memory.

To use the app, see design notes (in general single keys on keyboard access menu/submenus, e.g P for Page menu)

Regards
rpalmer

[TotO]

Nice begin.
Why not porting Lynx web browser for the CPC?


http://en.wikipedia.org/wiki/Lynx_(web_browser)

[rpalmer]

TotO,

I have indeed downloaded the C source for a version of the lynx browser, but the code is far too large to be built in one step and the resulting executable would not be fast enough for the CPC even it were compiled with existing compilers. Also the the source has many C particulars which the CPC compilers cannot handle.

The lynx web browser also has a graphical front-end to handle fonts of various sizes.

My version has a limited set of fonts available and also the browser is at first a beta version to get the most basic of functions up.

I cannot yet get the CPC to handle style-sheets, java/javascript and many more features of HTML V4.0.

For now I intend to get the browser to handle:

1. Simple text
2. Hyperlinks
3. Tables
4. Frames (limit to 1 for now).

As I develop the browser there will no doubt be enhancements and improvements to get more features into it.

I have already updated the version I previously uploaded where there are now more menu options available to see.  Once all menu options work as expected I will again upload this version so that people can provide comments. Once all of the menus are complete this leaves me with the last thing on the list of beta functions to work on which is the loading and rendering of the HTML.

I hope that when this app is running I will no doubt like to hear about what people like to see in the browser.

rpalmer

[Bryce]

That's some impressive goals you've set yourself, I hope you reach them. I'm looking forward to trying it out, just to see what our good old CPC is actually capable of.

Bryce.

[Gryzor]

Any screenies?

[rpalmer]

Gryzor,

I have managed to get all the menu options completed (except for some web-page specific ones).

I will soon post the executable version in a .dsk version so everyone can play with the version.

I have yet to get the HTML to load and view as I am working on that part right now.

rpalmer

[Optimus]

That sounds great.
I just didn't have the luck to run it properly under Winape32.


The Winape32 version I have has configuration of CPC6128 with Parados.
I don't know what proper positions to place ROMS are.
But I tried UPPER 1 and UPPER 2
Also I tried UPPER 8 and UPPER 9 (just after PARADOS)


The web browser loads and I see a glimpse of it, but then I get a sudden reset. Pausing the emulator just before the crash I see a "bad command"

[TFM]

I have yet to get the HTML to load and view as I am working on that part right now.

Well, a question for me would be if it is important to 'understand' the complete HTML language, or can you just work well with a subset of the HTML commands?

Is there a list of HTML commands in the net? I mean a list of important commands that should be implemented in any browser?

[mr_lou]

As I suggested earlier regarding a CPC browser, I'll suggest again:

Do it like Opera Mini does. The CPC browser fetches all pages from a specific server dedicated to the CPC browser. This server will fetch the real page from the actual server, and transform it into CPC code.

I.e., you type www.google.com
The CPC-browser tells the CPC-browser-server (at some IP address) to fetch www.google.com and transform it into something more lightweight. Reduces traffic.
The transformation could simply be converting HTML to some simpler format displayed by the browser. But could also just be a simpler HTML format.

[rpalmer]

TFM/FS,

There is a PDF of HTML 4.0 which details the whole specification.  I have downloaded it.  The browser will only handle a subset of the commands as they are quite difficult to get working (such as javascript for the <script> command). I have attached the text version.

Optimus,

I am uncertain why the version did not work, but will look into it.  At the moment the WinAPE i have configured does not use ParaDOS (only standard AMSDOS).
The version is had running was to have TCP/IP rom in slot 4 with HTML rom in slot 3. The roms themselves don't care which slot either is in and the browser has not hard wired ROM numbers.

mr_lou,

The whole purpose of the browser was to be able to get the pages direct from the website, however a CPC-browser server can be used should someone wish to create it. The CPC browser server is in essence similar to a secure VPN which pre-processes encrypted traffic outside of the browser.

The CPC browser will NOT be capable of handling graphics as it would slow down the CPC to such an extent that usability would become an issue.  However this first version is just to get the whole ball rolling at least and no doubt the future versions may make the graphics possible.

[TFM]

Aha! Thanks' a lot.

I remember when people talked that HTML is outdated and that everyone will move to XML or whatever. Now it seems like we still stick with HTML. Actually a lot of 8 bit relevant pages seem to be pretty close to 'just easy HTML' - some even excluding gfx.

[rpalmer]

Hello all,

I have checked my code and cannot seem to find out why someone had encountered a problem.

Alas, I have progressed on to complete the browser to now have a workable version to load only text files and to allow for scrolling. HTML loading is being developed as we speak.

The Find and Find Next options are not yet available as is handling fonts (but the browser can load up font files).

The attached zip contains the ROMs i used to configure my version of winape for tests.

My config for winape has

ROM 0 - OS6128
ROM 1 - BASIC1-1
ROM 3 - HTML
ROM 4 - TCPIP
ROM 5 - HDOSDR
ROM 7 - AMSDOS

I still have one issue, it's when the browser exits as it appears to lock up Winape and reboot option sometimes does not reboot.

If anyone tests this version, could you please let me know what you think of the interface and perhaps any suggestions to improve it.

rpalmer

[00WReX]

Just gave this a very quick try using WinCPC and it went fine...exit also functioned fine.

Good stuff.

Cheers,
Shane

[Gryzor]

I actually like the Opera mini approach. If someone could code a server we could host it here...


The browser itself could have a switch to, erm, switch it to 'direct' mode.

[rpalmer]

Gryzor,

I guess having a server to perform most of the file requests to the net is possible, but the CPC browser only handles test and no graphics. This means the a server will in essence do nothing more than what the browser does in the first place.

The design of the CPC browser to handle internet files is

a. Execute a request to download the HTML to a local file via the TCP/IP APIs.
b. Load the local file into the browser as is.
c. Render the file as per the HTML commands which the browser has implemented.

As you can see step 'a' has no gain in performance when a server does nothing more than get the file as the browser would still need to do the same thing from the server (in fact it may slow down the overall performance).

There would be a gain in performance if the CPC browser were to include graphic images, in which case the server could do the conversion to CPC screen formats much faster than what the cpc could ever do.

rpalmer

[mr_lou]

c. Render the file as per the HTML commands which the browser has implemented.

The problem there is, that webpages nowadays are usually very complex, stuffed with Javascript using Jquery libraries and plugins. Tables, stylesheets, Flash, HTML5 canvas stuff etc etc etc.
Your first thought is of course that you just avoid loading these things, but that's not always possible.
And then you have lots of different image formats, for the slow CPC to render.

By using a server you can give all the CPU demanding work to that. Have the server fetch the page, convert whatever images there might be into CPC image-bytes, arrange tables in a vertical structure, maybe even implement a few HTML5 canvas stuff. Then return this CPC-browser-optimized "CPC-html" code.
Doing it this way will sometimes be slower when going through the server - unless the page requested is already cached. But the time lost you will win back in the rendering process.

[Gryzor]

Mr_lou said it. IMO, even without images there's still lots of work to do in order to have a presentable page. It's not purely a matter of stripping images.


The current page, even without images, still weighs a hefty 186kB...

[ced64k]

You can set the user-agent to mobile to get a lightweight version of the websites (where this option exists, of course)

Like this : http://www.cpcwiki.eu/forum/?wap It's already more "compatible" with CPC 

[rpalmer]

It would be nice to have a user agent handle the brute force conversion, but for now lets get crawling before we start walking and then sprinting.

The browser configuration can handle pages of HTML up to 256K, so 186K is not beyond reach.

As for pages that have things like javascript and so on, these may well end-up needing a user-agent to get a complete viewable page.

I can't stress this enough that we need to get something working first and then improve on it to get more features into a useable version with modern web-pages.

rpalmer

[TFM]

Well, I know writing a Manual really suxx. But for anybody else (than the programmer) it is really a "must be" to have a manual. So I just want to encourage to do a comprehensive manual 

[mr_lou]

People with techno-joy never reads instructions.


Eddie Izzard's Encore on Computers

[robcfg]

Control, P, Print! Control, P, Print! 


Eddie Izzard is an awesone comedian, I have this show, 'Glorious', on DVD. My favorite is still 'Definite Article', hehe.