Category Archives: Android

Introducing OpenGL ES for Android: A Quick-Start Guide, Now in Beta

OpenGL ES for Android: A Quick-Start GuideHi all,

After nearly a year of working with the great team over at The Pragmatic Bookshelf, I am happy to announce that OpenGL ES for Android: A Quick-Start Guide is now in beta!

OpenGL ES for Android: A Quick-Start Guide follows in the tradition of this website and goes into more detail, teaching you how to create your first OpenGL project from scratch. You’ll learn about shaders, the OpenGL pipeline, and discover the power of OpenGL ES 2.0.  If you want to learn more about OpenGL for Android and you have a creative vision that you’d like to share with the world, then this is the book for you.

I would like to invite you, dear reader, to come check out the book and participate in the beta. As a beta book, the book is almost finished, with just a few rough edges and a couple of chapters to go, and a few things that need to be worked on. I could benefit greatly from your suggestions and feedback!

I hope that the book proves useful to you, and thanks for your continued support. :)

Share

OpenGL Community Roundup, January 2013 Edition

Welcome to another community round-up! A lot of people have been busy, creating great games, tutorials, and live wallpapers. Check them out below (in alphabetical order):

Advancing Usability

Ino

pleyasLab

Rene van der Lende

  • Kube — a tutorial on rotating a cube using OpenGL for Android.

Teaching Machines

Team Blubee

If I forgot someone or something, just let me know! I hope you all enjoyed the holidays, and I look forward to a great 2013, with more great stuff coming from the community. :)

Share

OpenGL Community Roundup, October Edition

It’s almost Halloween, which means it’s time to showcase another great set of apps and posts from the community. :)

First up, we have…

Wubbly Bubble Live Wallpaper

This interesting and lively live wallpaper renders a deforming 3D bubble to your home screen, with five different themes, many options for customization, and it runs on a wide range of devices. Download it here: https://play.google.com/store/apps/details?id=com.Ephemeral

Next up we have…

Blu Mountain Live Wallpaper

This is a very peaceful and relaxing live wallpaper, overlooking Mt. Fuji. You can watch the waves of the rhythmic water, and see the tranquil Sakura leaves fall gracefully as you overlook one of Japan’s most beloved national treasures.

This wallpaper supports the latest tablets, and works well with any Android theme or layout. It also features an upgrade to full HD art. Download it here: https://play.google.com/store/apps/details?id=com.blubee.wp

Roundup

Android OpenGL and slow rendering

Android terrain test

Happy Halloween, everyone!

Enhanced by Zemanta
Share

Understanding OpenGL’s Matrices

I often get questions related to OpenGL’s matrices: how do they work, how do they get built, and so forth. This is a topic that I have been frequently confused by, myself, and I feel that it warrants further explanation.

To better understand OpenGL’s matrices, and how and why we use them, we first need to understand the OpenGL coordinate space.

Normalized device coordinates

At the heart of things, OpenGL 2.0 doesn’t really know anything about your coordinate space or about the matrices that you’re using. OpenGL only requires that when all of your transformations are done, things should be in normalized device coordinates.

These coordinates range from -1 to +1 on each axis, regardless of the shape or size of the actual screen. The bottom left corner will be at (-1, -1), and the top right corner will be at (1, 1). OpenGL will then map these coordinates onto the viewport that was configured with glViewport. The underlying operating system’s window manager will then map that viewport to the appropriate place on the screen.

Adjusting to the screen’s aspect ratio

While OpenGL wants things to be in normalized device coordinates, it’s hard to work with these directly. The first problem is that they always range from -1 to +1, so if you use these coordinates directly, your image might be stretched when switching from portrait mode to landscape mode.

The first thing you can do to get around this problem is to define an orthographic projection. Android has the orthoM method; other platforms will have something similar. Let’s take a closer look at Android’s method:

orthoM(float[] m, int mOffset, float left, float right, float bottom, float top, float near, float far)

To define a simple matrix that adjusts things for the screen’s aspect ratio, we might call orthoM as follows:

float aspectRatio = (float) width / (float) height;
orthoM(projectionMatrix, 0, -aspectRatio, aspectRatio, -1, 1, -1, 1);

Let’s say that the screen dimensions are 800×600. The call would proceed as follows:

orthoM(projectionMatrix, 0, -1.333, 1.333, -1, 1, -1, 1);

Although the screen is wider than it’s tall, we automatically adjust the coordinate space to match by mapping -(800/600) to the left side and (800/600) to the right side.

This also works when we switch to portrait mode:

orthoM(projectionMatrix, 0, -0.75, 0.75, -1, 1, -1, 1);

We shrink the width in order to compensate for the smaller screen.

At the heart of things, the orthographic projection matrix will still convert things to the [-1, 1] range, since that’s what OpenGL expects. It just provides a way to adjust our coordinate space, so that we can see more of our scene if the screen is wider, and less if the screen is narrower.

3D projections

What about 3D projections? For those, we can use frustumM:

frustumM(float[] m, int offset, float left, float right, float bottom, float top, float near, float far)

We could define a simple 3D projection as follows:

frustumM(projectionMatrix, 0, -aspectRatio, aspectRatio, -1, 1, 1, 100);

The near & far range are handled differently: both have to be positive, and far has to be greater than near. We also have to watch out for the Z axis: frustumM will actually invert it, so that the negative Z points into the distance!

This has to do with convention: normalized device coordinates are in a left-handed coordinate system, while by convention, when we use a projection matrix, we work in a right-handed coordinate system.

Below is a good image illustrating the situation:

Left-handed and right-handed coordinate systems. Source: http://viz.aset.psu.edu/gho/sem_notes/3d_fundamentals/html/3d_coordinates.html

Left-handed and right-handed coordinate systems. Source: http://viz.aset.psu.edu/gho/sem_notes/3d_fundamentals/html/3d_coordinates.html

The perspective divide

The perspective projection doesn’t actually create the 3D effect; for that, we need to do something called the perspective divide. Each coordinate in OpenGL actually has four components, X, Y, Z, and W. The projection matrix sets things up so that after multiplying with the projection matrix, each coordinate’s W will increase the further away the object is. OpenGL will then divide by w: X, Y, Z will be divided by W. The further away something is, the more it will be pulled towards the center of the screen.

This PDF goes into more detail: http://www.terathon.com/gdc07_lengyel.pdf

This image shows how the same coordinate gets closer to the center of the screen as the W value increases:

Perspective divide by W

Perspective divide by W

Say you have three XYZ source positions of the following:

(3, 3,    -3)
(3, 3,    -6)
(3, 3, -1000)

 
The second point is a little bit further, or more “into the screen” than the first, and the third point is much further away than the second point. An infinite projection matrix would convert the coordinates as follows:

(3, 3,   1,    3)
(3, 3,   4,    6)
(3, 3, 998, 1000)

 
That last component is W. Now, OpenGL will divide everything by W, so you get something like this:

(1,     1,     0.33...)
(0.5,   0.5,   0.66...)
(0.003, 0.003, 0.998  )

 
There are two side effects of this division by W:

  1. The depth buffer becomes non-linear. There is a lot of Z precision up close, but less further away.
  2. If you try to do translations, etc… on your vertices after the perspective projection, then you won’t get the results you expect. This is because many of these operations depend on the W being 1, while after perspective projection it can be something else.

Here is an example of the perspective divide: imagine that we have a perspective projection matrix that looks as follows:

 1.5, 0,  0,    0,
 0,   1,  0,    0,
 0,   0, -1.2, -2.2,
 0,   0, -1,    0

 
This will transform coordinates as follows:

(1, 1, -1, 1) --> (1.5, 1, -1,   1)
(1, 1, -2, 1) --> (1.5, 1,  0.2, 2)
(2, 2, -2, 1) --> (3,   2,  0.2, 2)

 
After division by W, we get this:

(1.5, 1, -1,   1) --> (1.5,  1,   -1  )
(1.5, 1,  0.2, 2) --> (0.75, 0.5,  0.1)
(3,   2,  0.2, 2) --> (1.5,  1,    0.1)

 
Notice that the projection matrix just sets up the W, and it’s the actual divide by OpenGL that does the perspective effect. To verify this, try it out with a matrix calculator:

http://www.math.ubc.ca/~israel/applet/mcalc/matcalc.html

The view and model matrices

In OpenGL, we commonly use two additional matrices: the view and model matrices:

The model matrix

This matrix is used to move a model somewhere in the world. For example, let’s say we have a car model, and it’s defined such that it is centered around (0, 0, 0). We can place one car at (5, 5, 5) by setting up a model matrix that translates everything by (5, 5, 5), and drawing the car model with this matrix. We can then easily add a second car to the scene by just adjusting the translation. The model matrix helps us to push stuff out into the world.

The view matrix

The view matrix is functionally equivalent to a camera. It does the same thing as a model matrix, but it applies the same transformations equally to every object in the scene. Moving the whole world 5 units towards us is the same as if we had walked 5 units forwards.

Order of operations

These matrices all have to be multiplied in a specific way, if we want our results to be correct. Let’s start with some basic definitions:

vertexmodel
This is an original vertex, as defined inside one of our object models.
vertexworld
This is a vertex in world coordinates. We get to here by using a model matrix to push the model out into the world.
vertexeye
This is a vertex in eye coordinates. We get here by using a view matrix to move the entire scene around.
vertexclip
This is a vertex in clip coordinates (also known as homogeneous coordinates): this is the coordinate space after projection, but before the perspective divide.
vertexndc
This is a vertex in normalized device coordinates, and this is what we end up with after the perspective divide.

As we can see, getting to vertexndc is just a matter of applying each transformation in order. Let’s try to formulate this as an expression:

vertexndc = PerspectiveDivide(ProjectionMatrix * vertexeye)
vertexndc = PerspectiveDivide(ProjectionMatrix * ViewMatrix * vertexworld)
vertexndc = PerspectiveDivide(ProjectionMatrix * ViewMatrix * ModelMatrix * vertexmodel)

OpenGL takes care of the perspective division for us, so we don’t actually need to worry about that. All we need to worry about is the order of operations; since matrix multiplication is non commutative, we’ll get a different result depending on the order.

Column-major versus row-major order

A final point of confusion is often the layout of matrices in memory. OpenGL follows column-major order, meaning that the array offsets are specified like this:

0  4   8  12
1  5   9  13
2  6  10  14
3  7  11  15

 
m[0] … m[3] refer to the first column of the matrix.

Please let me know your questions, comments, and feedback!

Enhanced by Zemanta
Share
Spiral Maze!

Check Out These Cool Live Wallpapers, Games, and Tutorials by Fellow Learn OpenGL ESers!

The community has been busy here at Learn OpenGL ES, as many visitors have been developing fantastic games, live wallpapers, and tutorials of their own! I’d like to highlight some of the work that I have come across in recent weeks; if you’re part of the community here and would like to be featured as well, please give me a shout-out via the contact form!

Spiral Maze!

Spiral Maze!Buggies! - Live Wallpaper

Spiral Maze! is a fun and addictive little game by Prodigen, with physics implemented with JBox2D. Prodigen is also behind the highly ranked and regarded Buggies! – Live Wallpaper, as well as AppShaker. All are definitely worth checking out!

Paperland

Paperland Live WallpaperExodus Live WallpaperLight Grid Live Wallpaper

Then there’s Joko Interactive, the developer behind the incredibly beautiful and attractive Paperland Live WallpaperLight Grid Live WallpaperExodus Live Wallpaper, and many more!

Tutorials

I’d also like to give props to fellow community members Sankar and bluBee!

Sankar has been sharing his own work and tutorials here: http://code.google.com/p/my3dcube/

bluBee has been sharing his own work and tutorials here: http://code.google.com/p/blubee-opengles20/

Thanks for giving back to the community!

If you’re part of the community and would like to be featured as well, please let me know via the contact form!

Enhanced by Zemanta
Share
With one live wallpaper

How to Use OpenGL ES 2 in an Android Live Wallpaper

I’ve recently been looking into creating my first Live Wallpaper for Android as an opportunity to learn more about OpenGL ES and hopefully create something neat at the same time.

Throughout our entire series of Android tutorials, we’ve been using GLSurfaceView to take care of the hard work of threading and initialization for us; imagine my surprise when I learned that there is no equivalent for live wallpapers!

What we’re going to cover in this lesson

  • Adding multiple live wallpapers to a single APK.
  • Adding a thumbnail and description to the live wallpaper.
  • Using a GLSurfaceView to drive the live wallpaper.
  • Adapting the internals of GLSurfaceView to implement our own initialization and threading.
  • Allowing the user to switch between the two.

If you want to do OpenGL in a live wallpaper, it seems that there are a few alternatives:

  1. Do everything yourself. This might be a preferred option for some, but I won’t be exploring this in any more detail.
  2. Use GLSurfaceView, and adapt it to a live wallpaper. This lets us reuse a lot of code, but since this class was written to be used inside of an activity as a view, it might behave a little weirdly if we try to use it in a live wallpaper.
  3. Re-implement the features of GLSurfaceView. Robert Green has generously written up a re-implementation based on GLSurfaceView, and many have successfully based their wallpapers off of this code, so this might be one way to go. One caveat that I do have is that we need to duplicate a lot of the code and functionality inside GLSurfaceView, and we won’t be able to benefit from updates or variations between different versions of Android.

Since this is a research article, we’ll take a look at both options two and three. To start off, we’ll first look at adapting the existing GLSurfaceView for use within a live wallpaper. Later, we’ll see how we can use Robert Green’s work for rendering with OpenGL ES 2.0.

Here are the two main approaches we’re going to look at:

  • Approach 1: Using GLSurfaceView inside a live wallpaper
  • Approach 2: Using a custom live wallpaper based on the internals of GLSurfaceView

We’ll also look at combining the two approaches and allowing the user to select between the two. Let’s begin with the first approach: using GLSurfaceView.

Approach One: Using GLSurfaceView inside a live wallpaper

The first evidence I found that it was possible to use GLSurfaceView in a live wallpaper was from these files, by Ben Gruver:

I found these through this stack overflow question: http://stackoverflow.com/questions/4998533/android-live-wallpapers-with-opengl-es-2-0

Full credits to Ben Gruver for showing us one way that it could be done; in this article, we’ll work with our own adaptation of GLSurfaceView, with an emphasis on using it for OpenGL ES 2.0.

1. Creating the wallpaper service

Android Live Wallpapers are built off of WallpaperService, so the first thing we’ll do is create a new class called GLWallpaperService. Each service is really just a wrapper around an engine, which is subclassed from WallpaperService.Engine. This engine handles the actual lifecycle events of the live wallpaper.

We’ll be working with the Android OpenGL tutorials as a base, which can be downloaded from GitHub.

  1. Start off by creating a new package to contain the live wallpaper classes. For this lesson, I’ll be using com.learnopengles.android.livewallpaper.
  2. Inside this package, create a new class called GLWallpaperService.
Creating a new wallpaper service subclass

At the beginning of our new class, GLWallpaperService, extend WallpaperService:

public abstract class GLWallpaperService extends WallpaperService {
Creating a new wallpaper engine subclass

We’ll then create a new engine, so let’s also extend WallpaperService.Engine as an inner class:

public class GLEngine extends Engine {
Creating a custom subclass of GLSurfaceView

Inside the engine, create a new subclass of GLSurfaceView, called WallpaperGLSurfaceView:

class WallpaperGLSurfaceView extends GLSurfaceView {
	private static final String TAG = "WallpaperGLSurfaceView";

	WallpaperGLSurfaceView(Context context) {
		super(context);
	}

	@Override
	public SurfaceHolder getHolder() {
		return getSurfaceHolder();
	}

	public void onDestroy() {
		super.onDetachedFromWindow();
	}
}

We’ll be using this special subclass of GLSurfaceView to handle initializing OpenGL. The two methods to take note of are getHolder() and onDestroy(). Let’s take a look at these two methods in more detail:

getHolder()
Normally, GLSurfaceView calls getHolder() to get the surface holder provided by its superclass, SurfaceView, which assumes that it’s part of an activity as part of the content view. Since we’re driving a live wallpaper, we don’t want to use this surface. Instead, we override getHolder() to call [WallpaperService.Engine].getSurfaceHolder() to get the surface that’s associated with the live wallpaper.
onDestroy()
We need a way of telling the GLSurfaceView when it’s no longer valid. There’s no onDestroy(), but there is an onDetachedFromWindow() that we can use. We create our own onDestroy() to call onDetachedFromWindow().

Now that we have our extended GLSurfaceView in place, we can add a few lifecycle events and helper methods to round out a basic implementation:

Adding lifecycle events to the engine
private static final String TAG = "GLEngine";

private WallpaperGLSurfaceView glSurfaceView;
private boolean rendererHasBeenSet;

@Override
public void onCreate(SurfaceHolder surfaceHolder) {
	super.onCreate(surfaceHolder);
	glSurfaceView = new WallpaperGLSurfaceView(GLWallpaperService.this);
}

We initialize glSurfaceView in onCreate(). Remember that these methods should be placed inside of our inner class, GLEngine, and not at the same level as the wallpaper service. Let’s add a lifecycle event to handle visibility:

@Override
public void onVisibilityChanged(boolean visible) {
	super.onVisibilityChanged(visible);

	if (rendererHasBeenSet) {
		if (visible) {
			glSurfaceView.onResume();
		} else {
			glSurfaceView.onPause();			
		}
	}
}

Since we don’t have activity onPause() and onResume() events, we hook into the wallpaper engine’s onVisibilityChanged() event, instead. If the live wallpaper’s visible, we tell glSurfaceView to resume, and if it’s no longer visible, we tell it to pause.

We do check if the glSurfaceView is ready, since we shouldn’t call these methods unless a renderer has been set.

Let’s round out the life cycle events:

@Override
public void onDestroy() {
	super.onDestroy();
	glSurfaceView.onDestroy();
}

When the live wallpaper is destroyed, we tell glSurfaceView to stop rendering, using the custom onDestroy() method we defined earlier.

Adding helper methods to the engine

There’s also a couple of helper methods we’ll want to call from subclasses. Let’s define them now:

protected void setRenderer(Renderer renderer) {
	glSurfaceView.setRenderer(renderer);
	rendererHasBeenSet = true;
}

protected void setEGLContextClientVersion(int version) {
	glSurfaceView.setEGLContextClientVersion(version);
}

protected void setPreserveEGLContextOnPause(boolean preserve) {
	glSurfaceView.setPreserveEGLContextOnPause(preserve);
}

These methods simply wrap the glSurfaceView, so that we can call them from our subclasses.

2. Initializing OpenGL ES 2.0

The next step is to create a subclass that is derived from the GLWallpaperService that we’ve just created; this subclass will initialize OpenGL ES 2.0 and also initialize a custom renderer.

Create a new class, OpenGLES2WallpaperService, and add the following code:

public abstract class OpenGLES2WallpaperService extends GLWallpaperService {
	@Override
	public Engine onCreateEngine() {
		return new OpenGLES2Engine();
	}

	class OpenGLES2Engine extends GLWallpaperService.GLEngine {

		@Override
		public void onCreate(SurfaceHolder surfaceHolder) {
			super.onCreate(surfaceHolder);

			// Check if the system supports OpenGL ES 2.0.
			final ActivityManager activityManager =
				(ActivityManager) getSystemService(Context.ACTIVITY_SERVICE);
			final ConfigurationInfo configurationInfo =
				activityManager.getDeviceConfigurationInfo();
			final boolean supportsEs2 =
				configurationInfo.reqGlEsVersion >= 0x20000;

			if (supportsEs2)
			{
				// Request an OpenGL ES 2.0 compatible context.
				setEGLContextClientVersion(2);

				// On Honeycomb+ devices, this improves the performance when
				// leaving and resuming the live wallpaper.
				setPreserveEGLContextOnPause(true);

				// Set the renderer to our user-defined renderer.
				setRenderer(getNewRenderer());
			}
			else
			{
				// This is where you could create an OpenGL ES 1.x compatible
				// renderer if you wanted to support both ES 1 and ES 2.
				return;
			}
		}
	}

	abstract Renderer getNewRenderer();
}

As you can see, this is very similar to the OpenGL ES 2.0 initialization code that we’ve added to all of our activities in the Android tutorials. Calling setPreserveEGLContextOnPause() will ask the GLSurfaceView to preserve the OpenGL context, which will speed up pausing and restarting the live wallpaper by holding onto the surface.

3. Initializing a custom renderer

To round out the Java code, let’s add a final subclass, LessonThreeWallpaperService, which is going to use the renderer from lesson three as our live wallpaper:

public class LessonThreeWallpaperService extends OpenGLES2WallpaperService {
	@Override
	Renderer getNewRenderer() {
		return new LessonThreeRenderer();
	}
}

4. Adding the wallpaper to the manifest

Before we can install the live wallpaper on a device, we need to add it to the manifest and create a configuration file for it. Add the following to AndroidManifest.xml:

<service
    android:name=".livewallpaper.LessonThreeWallpaperService"
    android:label="@string/lesson_three_wallpaper_1"
    android:permission="android.permission.BIND_WALLPAPER" >
    <intent-filter>
        <action android:name="android.service.wallpaper.WallpaperService" />
    </intent-filter>

    <meta-data
        android:name="android.service.wallpaper"
        android:resource="@xml/wallpaper" />
</service>

This tells Android that this service is a live wallpaper, with the label set to @string/lesson_three_wallpaper_1. Let’s define a couple extra strings in strings.xml as follows:

<string name="lesson_three_wallpaper_1">Using GLSurfaceView</string>
<string name="lesson_three_wallpaper_2">Using re-implementation of GLSurfaceView</string>
<string name="lesson_three_wallpaper_3">Switching implementations</string>

We also need to create a wallpaper definition file. Create a new file, /res/xml/wallpaper.xml, and add the following:

<wallpaper xmlns:android="http://schemas.android.com/apk/res/android"
    android:thumbnail="@drawable/ic_lesson_three" />

This tells Android that this wallpaper should use this thumbnail when the user looks at the list of live wallpapers.

5. Build the application and view the wallpaper.

Now we can build the app and check out the live wallpaper. Install the app to your device, then go to your live wallpapers. You should see the following:

After selecting the live wallpaper, you should see a preview of lesson 3 on the screen:

Caveats with GLSurfaceView

Here are some of the caveats I can think of off of the top of my head; since this is a research article, I’d love to get your feedback.

  • I currently don’t understand enough about how SurfaceView works to see if using a GLSurfaceView uses additional resources in allocating Surfaces that never get used, since we’re using the live wallpaper’s surface.
  • SurfaceView, from which GLSurfaceView is based, sets the surface to PixelFormat.RGB_565 by default, whereas the WallpaperService sets PixelFormat.RGBX_8888 by default (checked against the ICS sources).

Approach Two: Using a custom live wallpaper based on the internals of GLSurfaceView

[Note: The wallpaper has changed since this was first written, as Mark Guerra and other contributors have extended Robert Green's work at this GitHub repository: https://github.com/GLWallpaperService/GLWallpaperService. The rest of this section is no longer required for adding support for OpenGL ES 2.0.]

As the second part of this research article, we’ll also be creating a live wallpaper with Robert Green and Mark Guerra’s adaptation of the code from GLSurfaceView. For this, you’ll want to download GLWallpaperService from http://git.io/gbhjmQ. Since we already have a GLWallpaperService, let’s create a new package to contain this class, and let’s call it com.learnopengles.android.rbgrnlivewallpaper.

After you have copied the class from GitHub, be sure to update the package name and any imports.

1. Adding support for OpenGL ES 2.0.

The first thing to note with this wallpaper service is that it doesn’t seem to have support for OpenGL ES 2.0. We’ll add this support by using the source code for GLSurfaceView as a base.

Open up the GLWallPaperService we just downloaded, and let’s make the following changes:

Adding mEGLContextClientVersion

The first thing we’ll do is add a new variable called mEGLContextClientVersion. Add the following to the beginning of GLEngine:

private int mEGLContextClientVersion;
Adding a method to set the EGL context client version

The next thing we’ll need to do is adapt setEGLContextClientVersion() from GLSurfaceView. Add the following method before setRenderer():

public void setEGLContextClientVersion(int version) {
    checkRenderThreadState();
    mEGLContextClientVersion = version;
}
Updating DefaultContextFactory

The next thing we need to do is to update createContext() inside the class DefaultContextFactory. Update the interface EGLContextFactory and the class DefaultContextFactory as follows:

interface EGLContextFactory {
	EGLContext createContext(EGL10 egl, EGLDisplay display, EGLConfig eglConfig, int eglContextClientVersion);

	void destroyContext(EGL10 egl, EGLDisplay display, EGLContext context);
}

class DefaultContextFactory implements EGLContextFactory {
	private int EGL_CONTEXT_CLIENT_VERSION = 0x3098;

	public EGLContext createContext(EGL10 egl, EGLDisplay display, EGLConfig config, int eglContextClientVersion) {
        int[] attrib_list = {EGL_CONTEXT_CLIENT_VERSION, eglContextClientVersion,
                EGL10.EGL_NONE };

        return egl.eglCreateContext(display, config, EGL10.EGL_NO_CONTEXT,
        		eglContextClientVersion != 0 ? attrib_list : null);
    }
Updating EglHelper

We’ll also need to fix the call to createContext(). Find the call to mEGLContextFactory.createContext() which is located inside the class EglHelper, and update it as follows:

mEglContext = mEGLContextFactory.createContext(mEgl, mEglDisplay,
			mEglConfig, mEGLContextClientVersion);

Since mEGLContextClientVersion has not been defined in this scope, add it to the beginning of EglHelper, and pass it in the constructor as follows:

private int mEGLContextClientVersion;

public EglHelper(EGLConfigChooser chooser,
		EGLContextFactory contextFactory,
		EGLWindowSurfaceFactory surfaceFactory, GLWrapper wrapper,
		int eglContextClientVersion) {
	...
	this.mEGLContextClientVersion = eglContextClientVersion;
}
Updating GLThread

Now GLThread doesn’t compile, so we’ll need to define mEGLContextClientVersion in that scope, too. Add the following code:

private int mEGLContextClientVersion;

GLThread(GLWallpaperService.Renderer renderer, EGLConfigChooser chooser,
		EGLContextFactory contextFactory,
		EGLWindowSurfaceFactory surfaceFactory, GLWrapper wrapper,
		int eglContextClientVersion) {
		...
		this.mEGLContextClientVersion = eglContextClientVersion;
	}

Now we can update the call to new EglHelper as follows:

mEglHelper = new EglHelper(mEGLConfigChooser, mEGLContextFactory,
				mEGLWindowSurfaceFactory, mGLWrapper, mEGLContextClientVersion);
Updating setRenderer()

Now that we’ve added the variables at the right scope levels, we can now adjust the call to new GLThread, and pass in the mEGLContextClientVersion from GLEngine. Update the call to new GLThread in setRenderer() as follows:

mGLThread = new GLThread(renderer, mEGLConfigChooser,
			mEGLContextFactory, mEGLWindowSurfaceFactory, mGLWrapper,
			mEGLContextClientVersion);
Updating BaseConfigChooser

There’s one more change we need to do before we can use setEGLContextClientVersion(). Update the class BaseConfigChooser as follows:

protected int mEGLContextClientVersion;

public BaseConfigChooser(int[] configSpec, int eglContextClientVersion) {
    this.mEGLContextClientVersion = eglContextClientVersion;
    mConfigSpec = filterConfigSpec(configSpec);
}

...

private int[] filterConfigSpec(int[] configSpec) {
    if (mEGLContextClientVersion != 2) {
        return configSpec;
    }
    /* We know none of the subclasses define EGL_RENDERABLE_TYPE.
     * And we know the configSpec is well formed.
     */
    int len = configSpec.length;
    int[] newConfigSpec = new int[len + 2];
    System.arraycopy(configSpec, 0, newConfigSpec, 0, len-1);
    newConfigSpec[len-1] = EGL10.EGL_RENDERABLE_TYPE;
    newConfigSpec[len] = 4; /* EGL_OPENGL_ES2_BIT */
    newConfigSpec[len+1] = EGL10.EGL_NONE;
    return newConfigSpec;
}
Updating ComponentSizeChooser

We’ll need to scope this new variable, so let’s update the constructor for ComponentSizeChooser:

public ComponentSizeChooser(int redSize, int greenSize, int blueSize,
		int alphaSize, int depthSize, int stencilSize, int eglContextClientVersion) {
	super(new int[] { EGL10.EGL_RED_SIZE, redSize,
			EGL10.EGL_GREEN_SIZE, greenSize, EGL10.EGL_BLUE_SIZE,
			blueSize, EGL10.EGL_ALPHA_SIZE, alphaSize,
			EGL10.EGL_DEPTH_SIZE, depthSize, EGL10.EGL_STENCIL_SIZE,
			stencilSize, EGL10.EGL_NONE }, eglContextClientVersion);
Updating SimpleEGLConfigChooser

We’ll need to keep scoping this variable in, so update SimpleEGLConfigChooser as follows:

public static class SimpleEGLConfigChooser extends ComponentSizeChooser {
	public SimpleEGLConfigChooser(boolean withDepthBuffer, int eglContextClientVersion) {
		super(4, 4, 4, 0, withDepthBuffer ? 16 : 0, 0, eglContextClientVersion);
Updating methods in GLEngine

Now that we’ve added the scoping, we’ll have to update our calls from our methods in GLEngine as follows:

mEGLConfigChooser = new SimpleEGLConfigChooser(true, mEGLContextClientVersion);
...
setEGLConfigChooser(new SimpleEGLConfigChooser(needDepth, mEGLContextClientVersion));
...
setEGLConfigChooser(new ComponentSizeChooser(redSize, greenSize,
					blueSize, alphaSize, depthSize, stencilSize, mEGLContextClientVersion));

2. Initializing OpenGL ES 2.0.

Now that we’ve updated GLWallpaperView to add support for OpenGL ES 2.0, we can now subclass it to initialize OpenGL ES 2.0. Let’s copy the same OpenGLES2WallpaperService from before into our new package, com.learnopengles.android.rbgrnlivewallpaper:

public abstract class OpenGLES2WallpaperService extends GLWallpaperService {
	@Override
	public Engine onCreateEngine() {
		return new OpenGLES2Engine();
	}

	class OpenGLES2Engine extends GLWallpaperService.GLEngine {

		@Override
		public void onCreate(SurfaceHolder surfaceHolder) {
			super.onCreate(surfaceHolder);

			// Check if the system supports OpenGL ES 2.0.
			final ActivityManager activityManager =
				(ActivityManager) getSystemService(Context.ACTIVITY_SERVICE);
			final ConfigurationInfo configurationInfo =
				activityManager.getDeviceConfigurationInfo();
			final boolean supportsEs2 =
				configurationInfo.reqGlEsVersion >= 0x20000;

			if (supportsEs2)
			{
				// Request an OpenGL ES 2.0 compatible context.
				setEGLContextClientVersion(2);

				// Set the renderer to our user-defined renderer.
				setRenderer(getNewRenderer());
			}
			else
			{
				// This is where you could create an OpenGL ES 1.x compatible
				// renderer if you wanted to support both ES 1 and ES 2.
				return;
			}
		}
	}

	abstract Renderer getNewRenderer();
}

3. Adding a renderer.

We can also copy LessonThreeWallpaperService from before, and use it as is:

public class LessonThreeWallpaperService extends OpenGLES2WallpaperService {
	@Override
	Renderer getNewRenderer() {
		return new LessonThreeRenderer();
	}
}

We have a compile error because GLWallpaperService doesn’t use the same renderer interface. Let’s go back to it and delete the following lines:

public interface Renderer extends GLSurfaceView.Renderer {

}

We’ll need to update some class references, so replace all instances of GLWallpaperService.Renderer with Renderer, and optimize imports. Our code should now compile.

4. Updating the manifest.

As before, we’ll need to update the manifest to add in the new live wallpaper. Add the following to AndroidManifest.xml:

<service
    android:name=".rbgrnlivewallpaper.LessonThreeWallpaperService"
    android:label="@string/lesson_three_wallpaper_2"
    android:permission="android.permission.BIND_WALLPAPER" >
    <intent-filter>
        <action android:name="android.service.wallpaper.WallpaperService" />
    </intent-filter>

    <meta-data
        android:name="android.service.wallpaper"
        android:resource="@xml/wallpaper" />
</service>

5. Build the application and view the wallpaper.

If everything went well, then you should now see two live wallpapers in the list, corresponding to the two that we’ve created:

When we select one of the wallpapers, we should see the following:

Taking the best of approaches one & two: switching between implementations in a single live wallpaper

We’ve now shown that we can use either implementation as a backing for our live wallpaper: either by using a GLSurfaceView directly with a few slight modifications, or by adapting Robert Green’s work (itself based off of the internals of GLSurfaceView) to support OpenGL ES 2.0.

What if we could switch between implementations, based on a user toggle? That could be a useful debugging feature: if ever a user has an issue with one implementation, they could always try the other.

To do this, we need a new service that will return either the first engine we created or the second, based on the configured setting. Let’s create a new package called com.learnopengles.android.switchinglivewallpaper, and let’s create a new class called, you guessed it, GLWallpaperService.

1. Updating GLWallpaperService to support switching

The first thing we’ll do is copy the entire class body of GLWallpaperService from com.learnopengles.android.livewallpaper into the new GLWallpaperService. The only difference is that we’ll rename the engine to GLSurfaceViewEngine.

The second thing we’ll do is copy the entire class body of GLWallpaperService from com.learnopengles.android.rbgrnlivewallpaper into the new GLWallpaperService; let’s call that engine RbgrnGLEngine.

We can update each engine to implement the following interface, so that we won’t have to duplicate code afterwards:

interface OpenGLEngine {
	void setEGLContextClientVersion(int version);

	void setRenderer(Renderer renderer);
}

The code is too long to paste in its entirety, so you can double check against the GitHub code here: https://github.com/learnopengles/Learn-OpenGLES-Tutorials/blob/master/android/AndroidOpenGLESLessons/src/com/learnopengles/android/switchinglivewallpaper/GLWallpaperService.java

2. Creating a new OpenGLES2WallpaperService to support switching

In com.learnopengles.android.switchinglivewallpaper, create a new OpenGLES2WallpaperService with the following code:

public abstract class OpenGLES2WallpaperService extends GLWallpaperService {
	private static final String SETTINGS_KEY = "use_gl_surface_view";

	@Override
	public Engine onCreateEngine() {
		if (PreferenceManager.getDefaultSharedPreferences(
				OpenGLES2WallpaperService.this).getBoolean(SETTINGS_KEY, true)) {
			return new ES2GLSurfaceViewEngine();
		} else {
			return new ES2RgbrnGLEngine();
		}
	}

	class ES2GLSurfaceViewEngine extends GLWallpaperService.GLSurfaceViewEngine {

		@Override
		public void onCreate(SurfaceHolder surfaceHolder) {
			super.onCreate(surfaceHolder);
			init(this);
		}
	}

	class ES2RgbrnGLEngine extends GLWallpaperService.RgbrnGLEngine {

		@Override
		public void onCreate(SurfaceHolder surfaceHolder) {
			super.onCreate(surfaceHolder);
			init(this);
		}
	}

	void init(OpenGLEngine engine) {
		// Check if the system supports OpenGL ES 2.0.
		final ActivityManager activityManager = (ActivityManager) getSystemService(Context.ACTIVITY_SERVICE);
		final ConfigurationInfo configurationInfo = activityManager
				.getDeviceConfigurationInfo();
		final boolean supportsEs2 = configurationInfo.reqGlEsVersion >= 0x20000;

		if (supportsEs2) {
			// Request an OpenGL ES 2.0 compatible context.
			engine.setEGLContextClientVersion(2);

			// Set the renderer to our user-defined renderer.
			engine.setRenderer(getNewRenderer());
		} else {
			// This is where you could create an OpenGL ES 1.x compatible
			// renderer if you wanted to support both ES 1 and ES 2.
			return;
		}
	}

	abstract Renderer getNewRenderer();
}

As you can see, this is similar to the previous versions, except that it’s been updated to support switching. We can copy LessonThreeWallpaperService over to the new package as-is.

3. Adding a settings activity

The next thing we want to do is add a settings activity. Create WallpaperSettings as follows:

public class WallpaperSettings extends PreferenceActivity {
	protected void onCreate(Bundle savedInstanceState) {
		super.onCreate(savedInstanceState);
		addPreferencesFromResource(R.xml.preferences);
	}
}
Creating the settings XML

We also need to create an XML for the settings. Create preferences.xml under /res/xml:

<?xml version="1.0" encoding="utf-8"?>
<PreferenceScreen xmlns:android="http://schemas.android.com/apk/res/android" >
    <CheckBoxPreference
        android:key="use_gl_surface_view"
        android:title="Use GLSurfaceView"
        android:defaultValue="true"/>
</PreferenceScreen>
Creating a new wallpaper XML

We’ll also need to create a new wallpaper.xml. Copy the existing wallpaper.xml to switching_wallpaper.xml, and update its content as follows:

<?xml version="1.0" encoding="utf-8"?>
<wallpaper xmlns:android="http://schemas.android.com/apk/res/android"
    android:settingsActivity="com.learnopengles.android.switchinglivewallpaper.WallpaperSettings"
    android:thumbnail="@drawable/ic_lesson_three" />
Updating the manifest

We’ll now need to update AndroidManifest.xml. Add the following:

<service
    android:name=".switchinglivewallpaper.LessonThreeWallpaperService"
    android:label="@string/lesson_three_wallpaper_3"
    android:permission="android.permission.BIND_WALLPAPER" >
    <intent-filter>
        <action android:name="android.service.wallpaper.WallpaperService" />
    </intent-filter>

    <meta-data
        android:name="android.service.wallpaper"
        android:resource="@xml/switching_wallpaper" />
</service>
<activity
    android:name=".switchinglivewallpaper.WallpaperSettings"
    android:exported="true"
    android:label="@string/lesson_three_wallpaper_3"
    android:permission="android.permission.BIND_WALLPAPER"
    android:theme="@style/WallpaperSettingsLight" >
</activity>
Adding new themes

We defined a new theme, so let’s add the following style definitions:

/res/values/styles.xml

https://github.com/learnopengles/Learn-OpenGLES-Tutorials/blob/master/android/AndroidOpenGLESLessons/res/values/styles.xml

/res/values-v11/styles.xml

https://github.com/learnopengles/Learn-OpenGLES-Tutorials/blob/master/android/AndroidOpenGLESLessons/res/values-v11/styles.xml

/res/values-v14/styles.xml

https://github.com/learnopengles/Learn-OpenGLES-Tutorials/blob/master/android/AndroidOpenGLESLessons/res/values-v14/styles.xml

These styles are adapted from the Theme.WallpaperSettings and Theme.Light.WallpaperSettings included with Android, but also updated so that they use the Holo theme on later versions of Android.

If you’re having trouble building the project after adding the new styles, change your build target to a newer version of Android (note that the app will still work on older versions, as they will simply ignore the XML).

4. Viewing the new live wallpaper

We can now build and run the application, and we’ll see our new live wallpaper in the list:

If we select the new one, “Switching implementations”, and open up the settings, we’ll see something like this:

We can use the toggle to change implementations; the toggle won’t take effect until we go back to the list of live wallpapers and then go back in to view the preview.

Wrapping up

I’d love to hear from those who have actually implemented live wallpapers, as well as from Robert Green and the Android guys if they ever come across this post! I just got this working with a few hours of research and development, but when it comes to this, I have nowhere near the same level of experience as some of the experts out there. I’d love to hear from you about the upsides and downsides of both using GLSurfaceView and of rolling out a custom derivative implementation based on the internals of GLSurfaceView.

Further exercises

Is there a better way to handle switching between implementations? How would you do it? Also, what would you do to continue animating the wallpaper even when the settings screen is displayed on top? Maybe when the settings screen appears you can grab a reference to the wallpaper and manually call its onVisibilityChanged()? You could also check for isPreview() directly in onVisibilityChanged(), but that will lead to problems when you exit the preview window.

I defined the settings key string in two places (once as a constant in Java, and once as an XML string); what would you do to avoid this duplication?

Get the source

The full source code for this lesson can be downloaded from the project site on GitHub. A compiled version of the lesson can also be downloaded directly from the Android Market:

QR code for link to the app on the Android Market.

Thanks for stopping by, and please feel free to check out the code and share your comments below.

Enhanced by Zemanta
Share
Shark

Check out These Cool Apps from an Early Student of Learn OpenGL ES!

An early student of Learn OpenGL ES, Miguel, has been working on some neat apps lately. Miguel was one of the first readers at Learn OpenGL ES, and we’ve exchanged at least a few emails back and forth about different topics, such as working with the camera through matrices. We were both learning a lot back then, and that sort of interaction really helped to strengthen my own personal knowledge of OpenGL.

Miguel hasn’t stopped learning, and he recently sent me over a link to his live wallpaper, “Shark 3D Kosmos”. I’m really happy with the progress he’s made so far; read on to learn more!

Shark 3D Kosmos Live Wallpaper

This wallpaper features a scary and realistic shark swimming on your home screen! The shark can be rendered in different modes, including holo mode, and you can select from several different camera modes, including one where the shark swims straight at you! The live wallpaper even includes features to save battery usage.

Link: https://play.google.com/store/apps/details?id=ksm.plylab.ksmwpp

Kosmos Galaxy 3D

This is a neat dynamic simulation of the attraction between the center of a galaxy and its surrounding stars. You can move the simulation around and add galaxies in realtime.

Link: https://play.google.com/store/apps/details?id=pleyasLab.KSMglx

I’m happy with how far Miguel has come since the first lessons here at Learn OpenGL ES. If you’ve also learned from the site and would like to showcase your work, let me know!

Share
Android Lesson Eight: An Introduction to Index Buffer Objects (IBOs)

Android Lesson Eight: An Introduction to Index Buffer Objects (IBOs)

Android Lesson Eight: An Introduction to Index Buffer Objects (IBOs)In our last lesson, we learned how to use vertex buffer objects on Android. We learned about the difference between client-side memory and GPU-dedicated memory, and the difference between storing texture, position and normal data in separate buffers, or altogether in one buffer. We also learned how to work around Froyo’s broken OpenGL ES 2.0 bindings.

In this lesson, we’ll learn about index buffer objects, and go over a practical example of how to use them. Here’s what we’re going to cover:

  • The difference between using vertex buffer objects only, and using vertex buffer objects together with index buffer objects.
  • How to join together triangle strips using degenerate triangles, and render an entire height map in a single rendering call.

Let’s get started with the fundamental difference between vertex buffer objects and index buffer objects:

Vertex buffer objects and index buffer objects

In the previous lesson, we learned that a vertex buffer object is simply an array of vertex data which is directly rendered by OpenGL. We can use separate buffers for each attribute, such as positions and colors, or we can use a single buffer and interleave all of the data together. Contemporary articles suggest interleaving the data and making sure it’s aligned to 4-byte boundaries for better performance.

The downside to vertex buffers come when we use many of the same vertices over and over again. For example, a heightmap can be broken down into a series of triangle strips. Since each neighbour strip shares one row of vertices, we will end up repeating a lot of vertices with a vertex buffer.

Diagram of a heightmap using triangle strips and a vertex buffer object.

You can see a vertex buffer object containing the vertices for two triangle strip rows. The order of vertices is shown, and the triangles defined by these vertices is also shown, when drawn using glDrawArrays(GL_TRIANGLE_STRIP, …).  In this example, we’re assuming that each row of triangles gets sent as a separate call to glDrawArrays(). Each vertex contains data as follows:

vertexBuffer = {
    // Position - Vertex 1
    0, 0, 0,
    // Color
    1,1,1
    // Normal
    0,0,1,
    // Position - Vertex 2
    1, 0, 0,
    ... 
}

As you can see, the middle row of vertices needs to be sent twice, and this will also happen for each additional row of the height map. As our height map gets larger, our vertex buffer could end up having to repeat a lot of position, color, and normal data and consume a lot of additional memory.

How can we improve on this state of affairs? We can use an index buffer object. Instead of repeating vertices in the vertex buffer, we’ll define each vertex once, and only once. We’ll refer to these vertices using offsets into this vertex buffer, and when we need to reuse a vertex, we’ll repeat the offset instead of repeating the entire vertex. Here’s a visual illustration of the new vertex buffer:

Vertex buffer object for use with an index buffer object.

Notice that our vertices are no longer linked together into triangles. We’ll no longer pass our vertex buffer object directly; instead, we’ll use an index buffer to tie the vertices together. The index buffer will contain only the offsets to our vertex buffer object. If we wanted to draw a triangle strip using the above buffer, our index buffer would contain data as follows:

indexBuffer = {
    1, 6, 2, 7, 3, 8, ...
}

That’s how we’ll link everything together. When we repeat the middle row of vertices, we only repeat the number, instead of repeating the entire block of data. We’ll draw the index buffer with a call to glDrawElements(GL_TRIANGLE_STRIP, …).

Linking together triangle strips with degenerate triangles

The examples above assumed that we would render each row of the heightmap with a separate call to glDrawArrays() or glDrawElements(). How can we link up each row to the next? After all, the end of the first row is all the way on the right, and the beginning of the second row on the left. How do we link the two?

We don’t need to get fancy and start drawing right to left or anything like that. We can instead use what’s known as a degenerate triangle. A degenerate triangle is a triangle that has no area, and when the GPU encounters such triangles, it will simply skip over them.

Let’s look at our index buffer again:

indexBuffer = {
    1, 6, 2, 7, 3, 8, 4, 9, ...
}

When drawing with GL_TRIANGLE_STRIP, OpenGL will build triangles by taking each set of three vertices, advancing by one vertex for each triangle. Every subsequent triangle shares two vertices with the previous triangle. For example, here are the sets of vertices that would be grouped into triangles:

  • Triangle 1 = 1, 6, 2
  • Triangle 2 = 6, 2, 7
  • Triangle 3 = 2, 7, 3
  • Triangle 4 = 7, 3, 8
  • Triangle 5 = 3, 8, 4
  • Triangle 6 = 8, 4, 9

OpenGL also maintains a specific order, or winding when building the triangles. The order of the first 3 vertices determines the order for the rest. If the first triangle is counter-clockwise, the rest will be counter-clockwise, too. OpenGL does this by swapping the first two vertices of every even triangle (swapped vertices are in bold):

  • Triangle 1 = 1, 6, 2
  • Triangle 2 = 2, 6, 7
  • Triangle 3 = 2, 7, 3
  • Triangle 4 = 3, 7, 8
  • Triangle 5 = 3, 8, 4
  • Triangle 6 = 4, 8, 9

Let’s show the entire index buffer, including the missing link between each row of the height map:

indexBuffer = {
    1, 6, 2, 7, 3, 8, 4, 9, 5, 10, ..., 6, 11, 7, 12, 8, 13, 9, 14, 10, 15
}

What do we need to put in between, in order to link up the triangles? We’ll need an even number of new triangles in order to preserve the winding. We can do this by repeating the last vertex of the first row, and the first vertex of the second row. Here’s the new index buffer below, with the duplicated vertices in bold:

indexBuffer = {
    1, 6, 2, 7, 3, 8, 4, 9, 5, 10, 10, 6, 6, 11, 7, 12, 8, 13, 9, 14, 10, 15
}

Here’s what the new sequence of triangles looks like:

  • Triangle 8 = 5, 9, 10
  • Triangle 9 (degenerate) = 5, 10, 10
  • Triangle 10 (degenerate) = 10, 10, 6
  • Triangle 11 (degenerate) = 10, 6, 6
  • Triangle 12 (degenerate) = 6, 6, 11
  • Triangle 13 = 6, 11, 7

By repeating the last vertex and the first vertex, we created four degenerate triangles that will be skipped, and linked the first row of the height map with the second. We could link an arbitrary number of rows this way and draw the entire heightmap with one call to glDrawElements(). Let’s take a look at this visually:

Index buffer object for height map with degenerate triangles.

The degenerate triangles link each row with the next row.

Degenerate triangles done the wrong way

We need to repeat both the last vertex of the first row and the first of the second row. What would happen if we didn’t? Let’s say we only repeated one vertex:

indexBuffer = {
    1, 6, 2, 7, 3, 8, 4, 9, 5, 10, 10, 6, 11, 7, 12, 8, 13, 9, 14, 10, 15
}

Here’s what the sequence of triangles would look like:

  • Triangle 8 = 5, 9, 10
  • Triangle 9 (degenerate) = 5, 10, 10
  • Triangle 10 (degenerate) = 10, 10, 6
  • Triangle 11 = 10, 6, 11
  • Triangle 12 = 11, 6, 7

Triangle 11 starts at the right and cuts all the way across to the left, which isn’t what we wanted to happen. The winding is now also incorrect for the next row of triangles, since 3 new triangles were inserted, swapping even and odd.

A practical example

Let’s walk through the code to make it happen. I highly recommend heading over and reading Android Lesson Seven: An Introduction to Vertex Buffer Objects (VBOs) before continuing.

Do you remember those graph calculators, that could draw parabolas and other stuff on the screen? In this example, we’re going to draw a 3d parabola using a height map. We’ll walk through all of the code to build and draw the height map. First, let’s get started with the definitions:

class HeightMap {
	static final int SIZE_PER_SIDE = 32;
	static final float MIN_POSITION = -5f;
	static final float POSITION_RANGE = 10f;

	final int[] vbo = new int[1];
	final int[] ibo = new int[1];

	int indexCount;
  • We’ll set the height map to 32 units per side, for a total of 1,024 vertices and 1,922 triangles, not including degenerate triangles (The total number of triangles in a height map is equal to (2 * (units_per_side – 1)2)). The height map positions will range from -5 to +5.
  • The OpenGL reference to our vertex buffer object and index buffer object will go in vbo and ibo, respectively.
  • indexCount will hold the total number of generated indices.
Building the vertex data

Let’s look at the code that will build the vertex buffer object. Remember that we still need a place to hold all of our vertices, and that each vertex will be defined once, and only once.

HeightMap() {
	try {
		final int floatsPerVertex = POSITION_DATA_SIZE_IN_ELEMENTS + NORMAL_DATA_SIZE_IN_ELEMENTS
				+ COLOR_DATA_SIZE_IN_ELEMENTS;
		final int xLength = SIZE_PER_SIDE;
		final int yLength = SIZE_PER_SIDE;

		final float[] heightMapVertexData = new float[xLength * yLength * floatsPerVertex];

		int offset = 0;

		// First, build the data for the vertex buffer
		for (int y = 0; y &lt; yLength; y++) {
			for (int x = 0; x &lt; xLength; x++) {
				final float xRatio = x / (float) (xLength - 1);

				// Build our heightmap from the top down, so that our triangles are 
				// counter-clockwise.
				final float yRatio = 1f - (y / (float) (yLength - 1));

				final float xPosition = MIN_POSITION + (xRatio * POSITION_RANGE);
				final float yPosition = MIN_POSITION + (yRatio * POSITION_RANGE);

				...
			}
		}

This bit of code sets up the loop to generate the vertices. Before we can send data to OpenGL, we need to build it in Java’s memory, so we create a floating point array to hold the height map vertices. Each vertex will hold enough floats to contain all of the position, normal, and color information.

Inside the loop, we calculate a ratio between 0 and 1 for x and y. This xRatio and yRatio will then be used to calculate the current position for the next vertex.

Let’s take a look at the actual calculations within the loop:

// Position
heightMapVertexData[offset++] = xPosition;
heightMapVertexData[offset++] = yPosition;
heightMapVertexData[offset++] = ((xPosition * xPosition) + (yPosition * yPosition)) / 10f;

First up is the position. Since this is a 3D parabola, we’ll calculate the Z as X2 + Y2. We divide the result by 10 so the resulting parabola isn’t so steep.

// Cheap normal using a derivative of the function.
// The slope for X will be 2X, for Y will be 2Y.
// Divide by 10 since the position's Z is also divided by 10.
final float xSlope = (2 * xPosition) / 10f;
final float ySlope = (2 * yPosition) / 10f;

// Calculate the normal using the cross product of the slopes.
final float[] planeVectorX = {1f, 0f, xSlope};
final float[] planeVectorY = {0f, 1f, ySlope};
final float[] normalVector = {
		(planeVectorX[1] * planeVectorY[2]) - (planeVectorX[2] * planeVectorY[1]),
		(planeVectorX[2] * planeVectorY[0]) - (planeVectorX[0] * planeVectorY[2]),
		(planeVectorX[0] * planeVectorY[1]) - (planeVectorX[1] * planeVectorY[0])};

// Normalize the normal
final float length = Matrix.length(normalVector[0], normalVector[1], normalVector[2]);

heightMapVertexData[offset++] = normalVector[0] / length;
heightMapVertexData[offset++] = normalVector[1] / length;
heightMapVertexData[offset++] = normalVector[2] / length;

Next up is the normal calculation. As you’ll remember from our lesson on lighting, the normal will be used to calculate lighting. The normal of a surface is defined as a vector perpendicular to the tangent plane at that particular point. In other words, the normal should be an arrow pointing straight away from the surface. Here’s an visual example for a parabola:

Parabola with normals

The first thing we need is the tangent of the surface. Using a bit of calculus, (don’t worry, I didn’t remember it either and went and searched for an online calculator ;)) we know that we can get the tangent, or the slope from the derivative of the function. Since our function is X2 + Y2, our slopes will therefore be 2X and 2Y. We scale the slopes down by 10, since for the position we had also scaled the result of the function down by 10.

To calculate the normal, we create two vectors for each slope to define a plane, and we calculate the cross product to get the normal, which is the perpendicular vector.

We then normalize the normal by calculating its length, and dividing each component by the length. This ensures that the overall length will be equal to 1.

// Add some fancy colors.
heightMapVertexData[offset++] = xRatio;
heightMapVertexData[offset++] = yRatio;
heightMapVertexData[offset++] = 0.5f;
heightMapVertexData[offset++] = 1f;

Finally, we’ll set some fancy colors. Red will scale from 0 to 1 across the X axis, and green will scale from 0 to 1 across the Y axis. We add a bit of blue to brighten things up, and assign 1 to alpha.

Building the index data

The next step is to link all of these vertices together, using the index buffer.

// Now build the index data
final int numStripsRequired = yLength - 1;
final int numDegensRequired = 2 * (numStripsRequired - 1);
final int verticesPerStrip = 2 * xLength;

final short[] heightMapIndexData = new short[(verticesPerStrip * numStripsRequired)
		+ numDegensRequired];

offset = 0;

for (int y = 0; y &lt; yLength - 1; y++) { 	 	if (y &gt; 0) {
		// Degenerate begin: repeat first vertex
		heightMapIndexData[offset++] = (short) (y * yLength);
	}

	for (int x = 0; x &lt; xLength; x++) {
		// One part of the strip
		heightMapIndexData[offset++] = (short) ((y * yLength) + x);
		heightMapIndexData[offset++] = (short) (((y + 1) * yLength) + x);
	}

	if (y &lt; yLength - 2) {
		// Degenerate end: repeat last vertex
		heightMapIndexData[offset++] = (short) (((y + 1) * yLength) + (xLength - 1));
	}
}

indexCount = heightMapIndexData.length;

In OpenGL ES 2, an index buffer needs to be an array of unsigned bytes or shorts, so we use shorts here. We’ll read from two rows of vertices at a time, and build a triangle strip using those vertices. If we’re on the second or subsequent rows, we’ll duplicate the first vertex of that row, and if we’re on any row but the last, we’ll also duplicate the last vertex of that row. This is so we can link the rows together using degenerate triangles as described earlier.

We’re just assigning offsets here. Imagine we had the height map as shown earlier:

Vertex buffer object for use with an index buffer object.

With the index buffer, we want to end up with something like this:

Index buffer object for height map with degenerate triangles.

Our buffer will contain data like this:

heightMapIndexData = {1, 6, 2, 7, 3, 8, 4, 9, 5, 10, 10, 6, 6, 11, 7, 12, 8, 13, 9, 14, 10, 15}

Just keep in mind that although our examples start with 1, and go on to 2, 3, etc… in the actual code, our arrays should be 0-based and start with 0.

Uploading data into vertex and index buffer objects

The next step is to copy the data from Dalvik’s heap to a direct buffer on the native heap:

final FloatBuffer heightMapVertexDataBuffer = ByteBuffer
	.allocateDirect(heightMapVertexData.length * BYTES_PER_FLOAT).order(ByteOrder.nativeOrder())
	.asFloatBuffer();
heightMapVertexDataBuffer.put(heightMapVertexData).position(0);

final ShortBuffer heightMapIndexDataBuffer = ByteBuffer
	.allocateDirect(heightMapIndexData.length * BYTES_PER_SHORT).order(ByteOrder.nativeOrder())
	.asShortBuffer();
heightMapIndexDataBuffer.put(heightMapIndexData).position(0);

Remember, the index data needs to go in a short buffer or a byte buffer. Now we can create OpenGL buffers, and upload our data into the buffers:

GLES20.glGenBuffers(1, vbo, 0);
GLES20.glGenBuffers(1, ibo, 0);

if (vbo[0] &gt; 0 &amp;&amp; ibo[0] &gt; 0) {
	GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, vbo[0]);
	GLES20.glBufferData(GLES20.GL_ARRAY_BUFFER, heightMapVertexDataBuffer.capacity()
							* BYTES_PER_FLOAT, heightMapVertexDataBuffer, GLES20.GL_STATIC_DRAW);

	GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, ibo[0]);
	GLES20.glBufferData(GLES20.GL_ELEMENT_ARRAY_BUFFER, heightMapIndexDataBuffer.capacity()
							* BYTES_PER_SHORT, heightMapIndexDataBuffer, GLES20.GL_STATIC_DRAW);

	GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, 0);
	GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, 0);
} else {
	errorHandler.handleError(ErrorType.BUFFER_CREATION_ERROR, &quot;glGenBuffers&quot;);
}

We use GL_ARRAY_BUFFER to specify our vertex data, and GL_ELEMENT_ARRAY_BUFFER to specify our index data.

Drawing the height map

Much of the code to draw the height map will be similar as in previous lessons. I won’t cover the matrix setup code here; instead, we’ll just look at the calls to bind the array data and draw using the index buffer:

GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, vbo[0]);

// Bind Attributes
glEs20.glVertexAttribPointer(positionAttribute, POSITION_DATA_SIZE_IN_ELEMENTS, GLES20.GL_FLOAT,
		false, STRIDE, 0);
GLES20.glEnableVertexAttribArray(positionAttribute);

glEs20.glVertexAttribPointer(normalAttribute, NORMAL_DATA_SIZE_IN_ELEMENTS, GLES20.GL_FLOAT,
		false, STRIDE, POSITION_DATA_SIZE_IN_ELEMENTS * BYTES_PER_FLOAT);
GLES20.glEnableVertexAttribArray(normalAttribute);

glEs20.glVertexAttribPointer(colorAttribute, COLOR_DATA_SIZE_IN_ELEMENTS, GLES20.GL_FLOAT,
		false, STRIDE, (POSITION_DATA_SIZE_IN_ELEMENTS + NORMAL_DATA_SIZE_IN_ELEMENTS)
		* BYTES_PER_FLOAT);
GLES20.glEnableVertexAttribArray(colorAttribute);

// Draw
GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, ibo[0]);
glEs20.glDrawElements(GLES20.GL_TRIANGLE_STRIP, indexCount, GLES20.GL_UNSIGNED_SHORT, 0);

GLES20.glBindBuffer(GLES20.GL_ARRAY_BUFFER, 0);
GLES20.glBindBuffer(GLES20.GL_ELEMENT_ARRAY_BUFFER, 0);

In the previous lesson, we had to introduce a custom OpenGL binding to properly use VBOs, as they’re broken in Froyo. We’ll also need to use this binding for IBOs as well. Just like in the previous lesson, we bind our position, normal, and color data in our vertex buffer to the matching attribute in the shader, taking care to pass in the proper stride and start offset of each attribute. Both the stride and the start offset are defined in terms of bytes.

The main difference is when it’s time to draw. We call glDrawElements() instead of glDrawArrays(), and we pass in the index count and data type. OpenGL ES 2.0 only accepts GL_UNSIGNED_SHORT and GL_UNSIGNED_BYTE, so we have to make sure that we define our index data using shorts or bytes.

Rendering and lighting two-sided triangles

For this lesson, we don’t enable GL_CULL_FACE so that both the front and the back sides of triangles are visible. We need to make a slight change to our fragment shader code so that lighting works properly for our two-sided triangles:

float diffuse;

if (gl_FrontFacing) {
    diffuse = max(dot(v_Normal, lightVector), 0.0);
} else {
    diffuse = max(dot(-v_Normal, lightVector), 0.0);
}

We use the special variable gl_FrontFacing to find out if the current fragment is part of a front-facing or back-facing triangle. If it’s front-facing, then no need to do anything special: the code is the same as before. If it’s back-facing, then we simply invert the surface normal (since the back side of a triangle faces the opposite direction) and proceed with the calculations as before.

Further exercises

When would it be a downside to use index buffers? Remember, when we use index buffers the GPU has to do an additional fetch into the vertex buffer object, so we’re introducing an additional step.

Wrapping up

The full source code for this lesson can be downloaded from the project site on GitHub. A compiled version of the lesson can also be downloaded directly from the Android Market:

QR code for link to the app on the Android Market.

Thanks for stopping by, and please feel free to check out the code and share your comments below.

Enhanced by Zemanta
Share

Android Emulator Now Supports Native OpenGL ES2.0!

For those of you who may have missed it, the Android team recently released an upgraded emulator image that has support for native OpenGL ES 2.0! That’s right; not only is it now supported, but the calls are translated to desktop OpenGL so they can be accelerated by your native graphics layer. It’s still better to test on an actual device, but this is a great boon for those who want to try out their app on different form factors without having to go out and buy a bunch of tablets and devices.

Check out the video:

Three notes about the GPU emulation

  1. First, you need to edit your emulator image, go down to the hardware section, and add “GPU Emulation” and set it to true.
  2. Second, there’s a bug with the emulator such that this line: “final boolean supportsEs2 = configurationInfo.reqGlEsVersion >= 0×20000;” does not work. It will always return false. You can add “|| Build.FINGERPRINT.startsWith(“generic”)” or simply comment out these checks and assume that OpenGL ES 2 is supported, when running on the emulator.
  3. Finally, if it crashes with “no config found”, try adding this line before the call to “setRenderer(…)”: “glSurfaceView.setEGLConfigChooser(8 , 8, 8, 8, 16, 0);”

A quick roundup

Hope that everyone is enjoying the gradually improving weather out there.

Share

OpenGL ES Roundup, March 9, 2012

Thank yous and mentions

A big thank you goes to the following guys who have recently kindly featured Learn OpenGL ES:

Thanks again; you all have my deepest gratitude!

State of WebGL on Android

The following browsers now support WebGL on Android:

Using Vertex Buffer Objects on Froyo (Android 2.2, SDK 8)

Bindings can be downloaded from here:

I cover how to use these bindings in Android Lesson Seven: An Introduction to Vertex Buffer Objects (VBOs). Note: These bindings are NOT needed if you don’t mind excluding a quarter of the market (as of the time of this writing) and target Gingerbread and higher.

The new iPad

In non-Android related news, Apple just released the third generation iPad, known as the “new iPad”, with a 2048×1536 “retina display”. Here are the specs:

Apple A5 SoC (system on a chip):

Judging by initial benchmarks, this thing just screams. iOS had never had the “Android lag” problem seen so often on Android phones and tablets, and this new tablet will only continue that trend. To be fair, throw enough hardware at the problem and maybe even Android will become lag-free.

What are your thoughts on the new iPad? Interested in picking it up? I love competition, because the more Apple, Android, and Microsoft duke it out, the better it is for us consumers! So long as nobody ends up dominating the market.

Have a good weekend, all!

Enhanced by Zemanta
Share