WebGL中图片多级处理(FrameBuffer)

WebGL中图片多级处理(FrameBuffer),

在webgl的使用过程中,我们通常会想对texture进行多级处理并对其贴在表面显示

如对较精准的边缘检测,要先后使用灰度shader、模糊shader、边缘shader来进行处理,而每次的处理对象则是上一次处理后的texture,这就要对处理后的结果进行覆盖保存。

 

在众多webgl库中,直接有选项rederToTarget来实现将shader处理后的texture渲染并覆盖原texture,其是怎么完成这个步骤的呢?

这就要引出本文的主角——FrameBuffer

 

在webgl的使用过程中,我们通常会想对texture进行多级处理并对其贴在表面显示

FrameBuffer是什么


 

FBO(Frame Buffer Object)是被推荐用于将数据渲染到纹理对象的扩展。

FrameBuffer就像是一个webgl显示容器一样,平时我们使用gl.drawArrays或者gl.drawElements都是将对象绘制在了默认的窗口中,而当我们指定一个FrameBuffer为当前窗口时,则用这两个方法去绘制,则会将对象绘制于指定的FrameBuffer中。

 

如对较精准的边缘检测,要先后使用灰度shader、模糊shader、边缘shader来进行处理,而每次的处理对象则是上一次处理后的texture,这就要对处理后的结果进行覆盖保存。

FrameBuffer的使用


 

internalformat, int x, int y, sizei width,
sizei height, int border);
target: TEXTURE_2D, TEXTURE_

FBO的创建:

//创建一个Framebuffer
var fb = gl.createFramebuffer();
//将fb绑定为目前的窗口   
gl.bindFramebuffer(gl.FRAMEBUFFER,fb);

 

这样,我们则创建了一个新的可以绘制的buffer了,且其并不会被显示出来

但是,这样就可以了吗?我们想到的是将经过shader渲染后的texture渲染出来并交给下一个shader,这时则引入方法framebufferTexture2D

Reference from《OpenGL ES Reference Pages about FramebufferTexture2D》:
 
To render directly into a texture image, a specified image from a texture object can be attached as one of the logical buffers of the currently bound framebuffer object by calling the command
为了直接渲染至纹理图片中,一个纹理对象中指定的图片可用下面的方法绑定在当前使用的FBO上一个逻辑缓存中
void FramebufferTexture2D( enum target, enum attachment, enum textarget, uint texture, int level );
 
target:
• FRAMEBUFFER
attachment:
• If attachment is COLOR_ATTACHMENT0, then image must have a colorrenderable internal format.(色彩)
• If attachment is DEPTH_ATTACHMENT, then image must have a depthrenderable internal format.(深度)
• If attachment is STENCIL_ATTACHMENT, then image must have a stencilrenderable internal format.(模板)
textarget:
• TEXTURE_2D    (two-dimensional texture)
• TEXTURE_CUBE_MAP_POSITIVE_X  (three-dimensional +x texture)
• TEXTURE_CUBE_MAP_POSITIVE_Y  (three-dimensional +y texture)
• TEXTURE_CUBE_MAP_POSITIVE_Z  (three-dimensional +z texture)
• TEXTURE_CUBE_MAP_NEGATIVE_X  (three-dimensional -x texture)
• TEXTURE_CUBE_MAP_NEGATIVE_Y  (three-dimensional -y texture)
• TEXTURE_CUBE_MAP_NEGATIVE_Z  (three-dimensional -z texture)
texture:
  texture object
level:
  specifies the mipmap level of the texture image to be attached to the framebuffer and must be 0.

 

我们使用这个方法来进行绑定(本文只介绍色彩的绑定,尝试和模板类似,但是有不同之处,不在此讨论)

//创建一个纹理对象
var texture = gl.createTexture();
//使用如下的设置来创建texture,这样对texture的设置可以使我们对任何尺寸的图片进行处理
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);


var fb = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER,fb);
//使用该方法将texture的颜色值与FBO进行绑定
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, texture, 0);

 

绑定后,当我们执行gl.drawArrays或gl.drawElements方法时,则会将直接渲染至目前绑定的FBO上,而FBO又与texture的色彩进行了绑定,所以绘制时则也将色彩渲染至了texture中

这样,我们则可用两个FBO来进行队列加工:

OriginalImage –> texture1

texture1 –> gray –> texture2

texture2 –> blur –> texture1

texture1 –> edge –> texture2

 

下面是具体实现过程

var FBOs = [],
    textures = [];

for(var i = 0; i < 2; i++){
    var texture = gl.createTexture();
    gl.bindTexture(gl.TEXTURE_2D, texture);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

    var fb = gl.createFramebuffer();
    gl.bindFramebuffer(gl.FRAMEBUFFER,fb);
    gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, texture, 0);

    //store corresponding texture and fb
    textures.push(texture);
    FBOs.push(fb);
}

gl.bindTexture(gl.TEXTURE_2D, originalImageTexture);

for(var i = 0; i < 3; i++){
    switch(i){case 0:
            //set gray shader to current shader program
            //handle arguments to vs shader and fs shader
            break;
        case 1:
            //set blur shader to current shader program
            //handle arguments to vs shader and fs shader
            break;
        case 2:
            //set edge shader to current shader program
            //handle arguments to vs shader and fs shader
            break;
    }

    gl.bindFramebuffer(gl.FRAMEBUFFER, FBOs[i%2]);
    //set the viewport fits the images size
    gl.viewport(0, 0, imgWidth, imgHeight);
    gl.drawArrays(....); //or gl.drawElements(....);

    //set the rendered texture to current texture for next frambuffer using
    gl.bindTexture(gl.TEXTURE_2D, texture[i%2]);
}

 

完整的过程为:

originalTexture --> gray program --> set FBO1 --> draw --> FBO1 --> set texture1

texture1 --> blur program --> set FBO2 --> draw --> FBO2 --> set texture2

texture2 --> edge program --> set FBO1 --> draw --> FBO1 --> set texture1

 

该过程中,FBO1与texture1是进行色彩渲染绑定的,所以set
FBO1后进行渲染则会直接渲染至texture1

当我们完成了整个绘制的时候,要正常显示处理后的图片,则要从FBO中跳出来:

//set FBO to null to use default framebuffer
gl.bindFramebuffer(gl.FRAMEBUFFER, null);

 

这是我在做Polyer使用到的:

FrameBuffer的其它用处


 

在众多webgl库中,直接有选项rederToTarget来实现将shader处理后的texture渲染并覆盖原texture,其是怎么完成这个步骤的呢?

gl.readPixels

从FrameBuffer中读取像素颜色数据

Reference from 《webgl_2.0_reference_card》/《OpenGL ES Reference Pages about readPixels》:
 
Pixels in the current framebuffercan be read back into an ArrayBufferView object.
void readPixels(int x, int y, long width, long height,enum format, enum type, Object pixels)
 
x,y
• Specify the window coordinates of the first pixel that is read from the frame buffer. This location is the lower left corner of a rectangular block of pixels.
width,height
 
• Specify the dimensions of the pixel rectangle. width and height of one correspond to a single pixel.
format
• Specifies the format of the pixel data. The following symbolic values are accepted  RGBA in WebGL
type
• Specifies the data type of the pixel data. Must be UNSIGNED_BYTEin WebGL
pixels
  • Returns the pixel data.
 

 

在使用过程中,我们要先创建pixels对象来储存数据

//using ArrayBufferView to store pixels data only, Unit8Array is the best because each color data is a byte
var pixels = new Uint8Array(ImageWidth * ImageHeight * 4);

gl.readPixels(0, 0, ImageWidth, ImageHeight, gl.RGBA, gl.UNSIGNED_BYTE, pixels);

 

这样,我们则可以得到整个FBO中的色彩数据

 

这就要引出本文的主角——FrameBuffer

gl.CopyTexImage2D

 

gl.CopyTexSubImage2D

这两个函数都是用来从FBO中将数据复制至当前绑定的texture中的

CopyTexImage2D方法:

Reference from 《OpenGL ES Reference Pages about CopyTexImage2D》:
 
copy pixels into a 2D texture image
void CopyTexImage2D(enum target, int level,enum internalformat, int x, int y, sizei width,sizei height, int border);
 
target:
• TEXTURE_2D
• TEXTURE_CUBE_MAP_POSITIVE_{X, Y, Z},
• TEXTURE_CUBE_MAP_NEGATIVE_{X, Y, Z}
internalformat:
• ALPHA
• LUMINANCE
• LUMINANCE_ALPHA
• RGB
• RGBA
x,y
Specify the window coordinates of the lower left corner of the rectangular region of pixels to be copied.
width
Specifies the width of the texture image. Must be 0 or 2 n + 2 ⁡ border for some integer n.
height
Specifies the height of the texture image. Must be 0 or 2 m + 2 ⁡ border for some integer m.
border
Specifies the width of the border. Must be either 0 or 1.

 CopyTexSubImage2D方法:

Reference from 《OpenGL ES Reference Pages about CopyTexSubImage2D》:
 
copy a two-dimensional texture subimage
void CopyTexSubImage2D(enum target, int level, int xoffset,int yoffset, int x, int y, sizei width, sizei height);
 
target:
• TEXTURE_2D
• TEXTURE_CUBE_MAP_POSITIVE_{X, Y, Z},
• TEXTURE_CUBE_MAP_NEGATIVE_{X, Y, Z}
level:
Specifies the level-of-detail number. Level 0 is the base image level. Level n is the nth mipmap reduction image.
xoffset:
Specifies a texel offset in the x direction within the texture array.
yoffset:
Specifies a texel offset in the y direction within the texture array.
x,y:
Specify the window coordinates of the lower left corner of the rectangular region of pixels to be copied.
width:
Specifies the width of the texture subimage.
height:
Specifies the height of the texture subimage.

这两个方法的不同之处相信大家已经看得出来了

 

CopyTexSubImage2D相对CopyTexImage2D增加了offset来改变复制区域

其最终复制区域为:[x, xoffset + width – 1]与[y, yoffset + height
-1]。

 

而CopyTexImage2D则是比CopyTexSubImage2D多了internelformat参数来控制对像素数据复制的种类。

 

FrameBuffer是什么


 

FBO(Frame Buffer Object)是被推荐用于将数据渲染到纹理对象的扩展。

FrameBuffer就像是一个webgl显示容器一样,平时我们使用gl.drawArrays或者gl.drawElements都是将对象绘制在了默认的窗口中,而当我们指定一个FrameBuffer为当前窗口时,则用这两个方法去绘制,则会将对象绘制于指定的FrameBuffer中。

 

结语:

有了对texture灵活的操作,则我们才能做出更有趣的东西出来,而framebuffer在里面也是相当重要的一个角色。

FrameBuffer的使用


 

internalformat, int x, int y, sizei width,
sizei height, int border);
target: TEXTURE_2D, TEXTURE_

FBO的创建:

//创建一个Framebuffer
var fb = gl.createFramebuffer();
//将fb绑定为目前的窗口   
gl.bindFramebuffer(gl.FRAMEBUFFER,fb);

 

这样,我们则创建了一个新的可以绘制的buffer了,且其并不会被显示出来

但是,这样就可以了吗?我们想到的是将经过shader渲染后的texture渲染出来并交给下一个shader,这时则引入方法framebufferTexture2D

Reference from《OpenGL ES Reference
Pages about
FramebufferTexture2D》:

 

To render directly into a texture image, a specified image from a
texture object can be attached as one of the logical buffers of the
currently bound framebuffer object by calling the command

  • style=”color: #0000ff;”>为了直接渲染至纹理图片中,一个纹理对象中指定的图片可用下面的方法绑定在当前使用的FBO上一个逻辑缓存中*

void FramebufferTexture2D( enum target, enum attachment, enum
textarget, uint texture, int level );

* *

target:

• FRAMEBUFFER

attachment:

• If attachment is style=”color: #ff0000;”>COLOR_ATTACHMENT0, then image must
have a colorrenderable internal
format.(色彩)

• If attachment is style=”color: #ff0000;”>DEPTH_ATTACHMENT, then image must have
a depthrenderable internal
format.(深度)

• If attachment is style=”color: #ff0000;”>STENCIL_ATTACHMENT, then image must
have a stencilrenderable internal
format.(模板)

textarget:

• TEXTURE_2D     style=”color: #0000ff;”>(two-dimensional texture)

• TEXTURE_CUBE_MAP_POSITIVE_X style=”color: #000000;”>   style=”color: #0000ff;”>(three-dimensional +x
texture)

•  style=”color: #ff0000;”>TEXTURE_CUBE_MAP_POSITIVE_Y   style=”color: #0000ff;”>(three-dimensional +y
texture)

• TEXTURE_CUBE_MAP_POSITIVE_Z style=”color: #000000;”>   style=”color: #0000ff;”>(three-dimensional +z
texture)

• TEXTURE_CUBE_MAP_NEGATIVE_X style=”color: #000000;”>   style=”color: #0000ff;”>(three-dimensional -x
texture)

• TEXTURE_CUBE_MAP_NEGATIVE_Y  style=”color: #000000;”>  style=”color: #0000ff;”>(three-dimensional -y
texture)

• TEXTURE_CUBE_MAP_NEGATIVE_Z style=”color: #000000;”>   style=”color: #0000ff;”>(three-dimensional -z
texture)

texture:

*  texture object*

level:

*  specifies the mipmap level of the texture image to be attached
to the framebuffer and must be style=”color: #ff0000;”>0.*

 

我们使用这个方法来进行绑定(本文只介绍色彩的绑定,尝试和模板类似,但是有不同之处,不在此讨论)

//创建一个纹理对象
var texture = gl.createTexture();
//使用如下的设置来创建texture,这样对texture的设置可以使我们对任何尺寸的图片进行处理
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);


var fb = gl.createFramebuffer();
gl.bindFramebuffer(gl.FRAMEBUFFER,fb);
//使用该方法将texture的颜色值与FBO进行绑定
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, texture, 0);

 

绑定后,当我们执行gl.drawArrays或gl.drawElements方法时,则会将直接渲染至目前绑定的FBO上,而FBO又与texture的色彩进行了绑定,所以绘制时则也将色彩渲染至了texture中

这样,我们则可用两个FBO来进行队列加工:

OriginalImage –> texture1

texture1 –> gray –> texture2

texture2 –> blur –> texture1

texture1 –> edge –> texture2

 

下面是具体实现过程

var FBOs = [],
    textures = [];

for(var i = 0; i < 2; i++){
    var texture = gl.createTexture();
    gl.bindTexture(gl.TEXTURE_2D, texture);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
    gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);

    var fb = gl.createFramebuffer();
    gl.bindFramebuffer(gl.FRAMEBUFFER,fb);
    gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, texture, 0);

    //store corresponding texture and fb
    textures.push(texture);
    FBOs.push(fb);
}

gl.bindTexture(gl.TEXTURE_2D, originalImageTexture);

for(var i = 0; i < 3; i++){
    switch(i){case 0:
            //set gray shader to current shader program
            //handle arguments to vs shader and fs shader
            break;
        case 1:
            //set blur shader to current shader program
            //handle arguments to vs shader and fs shader
            break;
        case 2:
            //set edge shader to current shader program
            //handle arguments to vs shader and fs shader
            break;
    }

    gl.bindFramebuffer(gl.FRAMEBUFFER, FBOs[i%2]);
    //set the viewport fits the images size
    gl.viewport(0, 0, imgWidth, imgHeight);
    gl.drawArrays(....); //or gl.drawElements(....);

    //set the rendered texture to current texture for next frambuffer using
    gl.bindTexture(gl.TEXTURE_2D, texture[i%2]);
}

 

完整的过程为:

originalTexture --> gray program --> set FBO1 --> draw --> FBO1 --> set texture1

texture1 --> blur program --> set FBO2 --> draw --> FBO2 --> set texture2

texture2 --> edge program --> set FBO1 --> draw --> FBO1 --> set texture1

 

该过程中,FBO1与texture1是进行色彩渲染绑定的,所以set
FBO1后进行渲染则会直接渲染至texture1

当我们完成了整个绘制的时候,要正常显示处理后的图片,则要从FBO中跳出来:

//set FBO to null to use default framebuffer
gl.bindFramebuffer(gl.FRAMEBUFFER, null);

 

 

FrameBuffer的其它用处


 

附:

WebGL-1.0参考卡片:

OpenGL-ES-2.0参考卡片:

OpenGL-ES-2.0参考手册:

 

The end.

在webgl的使用过程中,我们通常会想对texture进行多级处理并对其贴在表面显示
如对较精准的边缘检测,…

gl.readPixels

从FrameBuffer中读取像素颜色数据

R*eference from
《webgl_2.0_reference_card》/[ style=”text-decoration: underline;”>《
OpenGL ES Reference Pages
about
readPixels》](

 

Pixels in the current framebuffercan be read back into
an ArrayBufferView object.

void readPixels(int x, int y, long
width, long height,enum format, enum type, Object pixels)

 

x,y

• Specify the window coordinates of the first pixel that is read from
the frame buffer. This location is the lower left corner of a
rectangular block of pixels.

width,height

 

• Specify the dimensions of the pixel
rectangle. width and height of one correspond to a single pixel.

format

• Specifies the format of the pixel data. The following symbolic
values are accepted  ``RGBAin
WebGL

type

• Specifies the data type of the pixel data. Must be style=”color: #ff0000;”>UNSIGNED_BYTEin WebGL

pixels

*  • Returns the pixel data.*

 

 

在使用过程中,我们要先创建pixels对象来储存数据

//using ArrayBufferView to store pixels data only, Unit8Array is the best because each color data is a byte
var pixels = new Uint8Array(ImageWidth * ImageHeight * 4);

gl.readPixels(0, 0, ImageWidth, ImageHeight, gl.RGBA, gl.UNSIGNED_BYTE, pixels);

 

这样,我们则可以得到整个FBO中的色彩数据

 

gl.CopyTexImage2D

gl.CopyTexSubImage2D

这两个函数都是用来从FBO中将数据复制至当前绑定的texture中的

CopyTexImage2D方法:

*Reference from [ style=”color: #0000ff;”>《OpenGL ES Reference Pages about
CopyTexImage2D》](

* *

copy pixels into a 2D texture image

void CopyTexImage2D(enum target, int level,enum internalformat, int
x, int y, sizei width,sizei height, int border);

 

target:

• TEXTURE_2D

• TEXTURE_CUBE_MAP_POSITIVE_{X, Y, Z},

• TEXTURE_CUBE_MAP_NEGATIVE_{X, Y, Z}

internalformat:

• ALPHA

• LUMINANCE

• LUMINANCE_ALPHA

• RGB

• RGBA

x,y

Specify the window coordinates of the lower left corner of the
rectangular region of pixels to be copied.

width

Specifies the width of the texture image. Must be 0
or 2 n + 2 ⁡ border for some integer n.

height

Specifies the height of the texture image. Must be 0
or 2 m + 2 ⁡ border for some integer m.

border

Specifies the width of the border. Must be either 0 or 1.

 CopyTexSubImage2D方法:

*Reference from  style=”color: #0000ff;”>《OpenGL ES Reference Pages about
CopyTexSubImage2D
》:*

* *

copy a two-dimensional texture subimage

void CopyTexSubImage2D(enum target, int level, int xoffset,int
yoffset, int x, int y, sizei width, sizei height);

* *

target:

• TEXTURE_2D

• TEXTURE_CUBE_MAP_POSITIVE_{X, Y, Z},

• TEXTURE_CUBE_MAP_NEGATIVE_{X, Y, Z}

level:

Specifies the level-of-detail number. Level 0 is the base image
level. Level n is the  class=”emphasis”>nth mipmap reduction image.

xoffset:

Specifies a texel offset in the x direction within the texture
array.

yoffset:

Specifies a texel offset in the y direction within the texture
array.

x,y:

Specify the window coordinates of the lower left corner of the
rectangular region of pixels to be copied.

width:

Specifies the width of the texture subimage.

height:

Specifies the height of the texture subimage.

这两个方法的不同之处相信大家已经看得出来了

 

CopyTexSubImage2D相对CopyTexImage2D增加了offset来改变复制区域

其最终复制区域为:[x, xoffset + width – 1]与[y, yoffset + height
-1]。

 

而CopyTexImage2D则是比CopyTexSubImage2D多了internelformat参数来控制对像素数据复制的种类。

 

结语:

有了对texture灵活的操作,则我们才能做出更有趣的东西出来,而framebuffer在里面也是相当重要的一个角色。

 

附:

WebGL-1.0参考卡片:

OpenGL-ES-2.0参考卡片:

OpenGL-ES-2.0参考手册:

 

The end.

相关文章