Web Browser from scratch

[tappatekie]

Hey guys it has been a while since I last posted on here.

I have a new programming project which is to make a web browser from scratch. I know this is an insanely huge project to undergo especially for a single developer, but I'm doing it for the sheer hell of it and I could learn so much in the process.

I have already coded the HTML/XML/XHTML parser as well as a CSS parser including all Pseudo classes. I just now have to do the boring bit of having border, background, text properties in CSS to be "understood" so the renderer can then be told what to render etc...

I'm now into the first stages of making the layout engine. But will be doing my own JavaScript engine last since it's not really important to do and it'l take the most time.

Is anyone here undergoing the same kind of project?

P.s It's in C#

[K-Bal]

Sounds ambitious If you get far with this it will be a good reference for programming jobs. Good luck!

[tappatekie]

Sounds ambitious If you get far with this it will be a good reference for programming jobs. Good luck!

Thank you And yeah I know it's insanely ambitious but I enjoy every second making it (except a few frustrating bits)

[Rebornxeno]

I was dabbling in this a bit when I started learning winsock, but I became really turned off when I had to start parsing html. Not that it was hard, but it wasn't very fun for me, and with other things I had wanted to do I dropped it. I don't think it's too ambitious, I think it's something any programmer is capable of and should probably do at some point, just to understand what goes on in such a widely used application. Many of the things you do in making a web browser fit into a wide array of applications, so you learn skills that carry over to a lot of different projects.

Are you making this open-source, so say someone like me could play around with? (i'd probably just take your html parser)

[tappatekie]

I was dabbling in this a bit when I started learning winsock, but I became really turned off when I had to start parsing html. Not that it was hard, but it wasn't very fun for me, and with other things I had wanted to do I dropped it. I don't think it's too ambitious, I think it's something any programmer is capable of and should probably do at some point, just to understand what goes on in such a widely used application. Many of the things you do in making a web browser fit into a wide array of applications, so you learn skills that carry over to a lot of different projects.

Are you making this open-source, so say someone like me could play around with? (i'd probably just take your html parser)

I am planning on making it open source and have it open to any contributions to the code (Like Google is doing with Chrome)

The HTML parser is actually a XML parser since it's got the same syntax (ish), just different tags. It was a bitch to get done, but it is now finished and have tested it on big websites like facebook, twitter and google and it processed it accurately (as well as testing it out on a few tag spaghetti websites).
Once I coded XML, I automatically had some support for HTML except the HTML parser puts all the tags in the correct place for the DOM e.g if you put a title tag in the body, it'l just move it to the head tag.

[tappatekie]

Quick update, I'm not going to touch the layout engine yet until I have a stable enough rendering base to work from.

The plan is to use an abstract class "ARenderer" which would hold All the functions needed for rendering e.g BeginFrame, EndFrame, Clear, FillPoly etc.. which the layout engine and form system would depend on to render stuff to the screen. This would leave it open to be cross platform (no plans yet) and removing as much dependency as possible from the .Net framework.

The renderer by default will be OpenGL based (thanks to Tao.OpenGL library for Mono and .Net)

An example of this use would be

Code: Select all

public abstract class ARenderer {
       public abstract void BeginFrame();
       public void Clear() { Clear(Color.Transparent); }
       public abstract void Clear(Color color);
       public abstract void EndFrame();
       public abstract IntPtr DeviceContext { get; }
}
public class Gdi32Renderer : ARenderer { 
       //vague implimentation that won't actually compile...
       public override void Clear(Color color) { gdiplusGraphics.Clear((GdiPlus.Color)color); }
}

You may notice the use of the "Color", this object is built into the browser core API and not using System.Drawing at all.

As of now, I am writing a window system that will eventually be open to cross platform if I ever wanted to go that way. It will be the same design as ARenderer so that I can easily create a Factory class to produce either LinuxWindow, X11Window or MSWindow etc... and the browser will generically handle the form given as if it was a windows form.

And no, I do not plan to use GDI+ under any circumstances unless there is really no other alternatives present e.g OpenGL or DirectX so it's kind of something to fallback on, but will dramatically reduce performance since everything will be done on the CPU.

[MarauderIIC]

As a word of advice, if you're using regexes (I don't know if you are or not), you're doing it wrong. HTML isn't regular.

[tappatekie]

As a word of advice, if you're using regexes (I don't know if you are or not), you're doing it wrong. HTML isn't regular.

Nope :P
It goes through every character in the string, detects a tag begin, gets the tag end, processes the string in the middle, gets where the tag ends then processes the string between the tag start and tag end (VERY simply put..)
It's not that slow, but the caching system would make it lightning fast.

[tappatekie]

I know I'm probably going to sound completely newbish... but I need help with OpenGL.

I've started making my own OpenGL wrapper class which takes in a window handle, gets the device context from it and creates a rendering context which then makes it current for OpenGL on the current thread. (It works beautifully by the way..)

The wrapped rendering class needs to inherit an abstract rendering class which only uses pixel coordinates, but the problem is that OpenGL only uses vertex float values (you know what I mean right?).
I need a way to convert pixel coords (I have view port size) into a value that OpenGL can translate correctly.

I have tried using glOrtho(0,Width,Height,0,0,1); but did'nt work at all as well as a few other things that I found online.

E.g So I can do this:

Code: Select all

public void FillRectangle(Color color, Rectangle rectangle) { 
        //just an example, this function converts the pixel positions in the rectangle to the appropriate OpenGL coordinates
        toGL(ref rectangle);

        Gl.GlBegin(Gl.GL_QUADS);
        Gl.GlColor4(color.A,color.R,color.G,color.B); //purposes of this, data types are'nt essential
        Gl.GlVertex3f(rectangle.X, rectangle.Y);
        Gl.GlVertex3f(rectangle.X+rectangle.Width, rectangle.Y);
        Gl.GlVertex3f(rectangle.X+rectangle.Width, rectangle.Y+rectangle.Height);
        Gl.GlVertex3f(rectangle.X, rectangle.Y+rectangle.Height);
        Gl.GlEnd();
}

P.S If you read my reply about 2 posts ago, I explain why I'm using abstracted rendering classes. Also this OpenGL render class will only be targeted for Win32 platforms, I will have alternative OpenGL rendering classes for it's appropriate platform e.g WinGL, XGL, LinuxGL.

This is my code currently to initialize OpenGL, it's only test code which is why it's sloppy but if your interested, this code will be performed by a PlatformFactory class (Win32Factory.CreateWindowRenderer(Win32Window window) and (Win32Factory.CreateBitmapRenderer(Bitmap bitmap)) which will be instance classes but will be exposed in a global static object "Factory" which will implement the correct platform and give out the appropriate render and window classes.

Code: Select all

            //create a pixel format
            Win32API.GDI32.PixelFormatDescriptor pxf = new Win32API.GDI32.PixelFormatDescriptor() {
                dwFlags = 0x00000004 | 0x00000020 | 0x00000001,
                          //drawwindow        supportogl   doublebuff        
                nVersion = 1,
                cColorBits = 32,
                cDepthBits = 16,
            }; pxf.nSize = (short)Marshal.SizeOf(pxf);
            int px = Win32API.GDI32.ChoosePixelFormat(p_DeviceContext, ref pxf);
            Win32API.GDI32.SetPixelFormat(p_DeviceContext, px, ref pxf);

            //create the rendering context with OpenGL
            IntPtr renderContext = wglCreateContext(p_DeviceContext);

            //make the window context, the current OpenGL context
            wglMakeCurrent(p_DeviceContext, renderContext);

            Gl.glClearColor(0, 0, 1, 1);
            Gl.glMatrixMode(Gl.GL_PROJECTION);
            Gl.glLoadIdentity();
            Gl.glOrtho(0, Width, Height, 0, 1, 1000);
            Gl.glClear(Gl.GL_COLOR_BUFFER_BIT);            
            Gl.glMatrixMode(Gl.GL_MODELVIEW);
            Gl.glLoadIdentity();
            Gl.glTranslatef(0, 0, -500);
            Gl.glMatrixMode(Gl.GL_PROJECTION);
            Gl.glLoadIdentity();
            Gl.glViewport(0, 0, Width, Height);
            Gl.glClear(Gl.GL_COLOR_BUFFER_BIT);
            Gl.glLoadIdentity();

[dandymcgee]

You might find this informative/useful: http://www.matrix44.net/cms/notes/openg ... -in-opengl

What you're trying to do is convert view coordinates to world coordinates. Googling something along those lines should give you plenty of information.

[qpHalcy0n]

Read up on what glLoadIdentity does. You've just replaced your projection after you've set it. All of this:

Code: Select all

            Gl.glClearColor(0, 0, 1, 1);
            Gl.glMatrixMode(Gl.GL_PROJECTION);
            Gl.glLoadIdentity();
            Gl.glOrtho(0, Width, Height, 0, 1, 1000);
            Gl.glClear(Gl.GL_COLOR_BUFFER_BIT);            
            Gl.glMatrixMode(Gl.GL_MODELVIEW);
            Gl.glLoadIdentity();
            Gl.glTranslatef(0, 0, -500);
            Gl.glMatrixMode(Gl.GL_PROJECTION);
            Gl.glLoadIdentity();
            Gl.glViewport(0, 0, Width, Height);
            Gl.glClear(Gl.GL_COLOR_BUFFER_BIT);
            Gl.glLoadIdentity();

is a fantastic mess.

[Rebornxeno]

I would offer my help if I could only make out what you are trying to ask.

which only uses pixel coordinates, but the problem is that OpenGL only uses vertex float values

Could you explain a little bit more about why this is a problem?

[MarauderIIC]

As a word of advice, if you're using regexes (I don't know if you are or not), you're doing it wrong. HTML isn't regular.

Nope :P
It goes through every character in the string, detects a tag begin, gets the tag end, processes the string in the middle, gets where the tag ends then processes the string between the tag start and tag end :) (VERY simply put..)
It's not that slow, but the caching system would make it lightning fast.

So <div id="<i like to put angle brackets in my ids>"> won't mess you up? ;)

[tappatekie]

So <div id="<i like to put angle brackets in my ids>"> won't mess you up?

No it won't if it comes across any " or ' characters, it seeks over the entire string and then looks for the <> again
I have tested this, and it does skip over both " and '.
Pratically anything you throw at it can be parsed correctly and even IF you have something like "\"" in there, it still does that correctly (the option is there in the code but currently it is disabled since it's not a standard HTML thing to allow "\"" but that's option is only used for CSS value processing

Read up on what glLoadIdentity does. You've just replaced your projection after you've set it. All of this:
is a fantastic mess.

I know it is a mess I've never touched on OpenGL libraries.

Could you explain a little bit more about why this is a problem?

It's a problem because I'm making an abstracted rendering class which has x,y,width,height passed as pixel coordinates on the screen so I can also impliment different rendering engines for different platforms etc...
The browser will just use the IRender inherited class passed to it from a PlatformFactory class so it doesn't need to care about touching OpenGL or Gdi+.

[Falco Girgis]

This is an insanely ambitious undertaking... I wish you luck!

I have tried using glOrtho(0,Width,Height,0,0,1); but did'nt work at all as well as a few other things that I found online.

That is how you do it. As qp mentioned, you are setting it up correctly, but then you're loading an identity matrix while still on the GL_PROJECTION matrix, which is basically erasing your glOrtho() call.