OpenCV for ZGE
Moderator: Moderators
OpenCV for ZGE
Based on discussion about new possible ZGE extensions, I created an experimental version of the ZgeCV library, a wrapper for OpenCV computer vision library.
The current demo version works only on Windows and just captures video from connected cameras and returns the data to be renderable by OpenGL. That's the first important step and next will come:
- allow for multiple camera capturing
- managing multiple frames/images to be processed independently
- import/export of images and videos from files
- various image/video filters
- object detection
- face recognition, etc.
OpenCV has a lot of possibilities, but at the other hand, an unpleasant aspect is its size. With the current video capturing features it is about 6.3 MB, when added some image processing features, it is about 12 MB. Just compare it to the compiled and uncompressed demo created in ZGE which is about 240 KB.
The library can be downloaded from here. The demo ZGE project is attached.
Your comments are welcome.
The current demo version works only on Windows and just captures video from connected cameras and returns the data to be renderable by OpenGL. That's the first important step and next will come:
- allow for multiple camera capturing
- managing multiple frames/images to be processed independently
- import/export of images and videos from files
- various image/video filters
- object detection
- face recognition, etc.
OpenCV has a lot of possibilities, but at the other hand, an unpleasant aspect is its size. With the current video capturing features it is about 6.3 MB, when added some image processing features, it is about 12 MB. Just compare it to the compiled and uncompressed demo created in ZGE which is about 240 KB.
The library can be downloaded from here. The demo ZGE project is attached.
Your comments are welcome.
- Attachments
-
- screenshot from camera capturing demo
- scr.jpg (29.14 KiB) Viewed 31684 times
-
- demo1.zgeproj
- demo ZGE project
- (2.52 KiB) Downloaded 999 times
Awsome
Thanks for doing this! I know opencv is time consuming to set-up. I'm glad you were able to get the project started. For the average computer user, OpenCV s not easy to get up and running - ZGE with OpenCV can make computer vision more accessible, and that is great.
Your camera demo is working perfect.
Computer Vision has so many applications, but be weary of face recognition, it's a very difficult problem. Face /object/shape detection (is there a face?) is simple with opencv, but recognition (who's face is it?) is complicated and not very accurate. If you start there, you'll get frustrated for sure.
Your camera demo is working perfect.
Computer Vision has so many applications, but be weary of face recognition, it's a very difficult problem. Face /object/shape detection (is there a face?) is simple with opencv, but recognition (who's face is it?) is complicated and not very accurate. If you start there, you'll get frustrated for sure.
Re: Awsome
This was not so difficult. The new version 3.0.0 beta comes with nice pre-built .dll and .lib files, so compilation and linking is quite an easy task. Also the API is relatively straightforward.StevenM wrote:Thanks for doing this! I know opencv is time consuming to set-up.
To avoid frustration from face recognition... (even if I do not expect bigger troubles than recognizing my face in a mirror by myself soon in the morning) I rather started with simpler things. The current update contains:
- support for several cameras simultaneously
- support for arbitrary number of images; this is useful for image operations
- an initial set of image operations/filters: color transformation, flipping, Gaussian blur, and Canny edge detection
Enjoy the new demo project and please download the latest dll to make it working. I wondered myself how OpenCV is fast and all the effects can be applied in runtime on captured video.
Next steps:
1. There is a lot of image operations in OpenCV, and I do not think I'll add all of them to ZgeCV library... if you have some preferences, just tell me.
2. I could try, for instance, object detection or motion analysis. StevenM do you have some experience with these topics? Could you please give me some good tutorials to be able to understand it fast and correctly?
- Attachments
-
- screenshot form new demo
- scr1.jpg (64.79 KiB) Viewed 31644 times
-
- demo2.zgeproj
- demo project
- (6.78 KiB) Downloaded 979 times
Re: Awsome
Hi Rado1,
*Not sure whether BitmapLoad works properly when you've adjusted the size of a Bitmap using glTexImage2D though.
K
Are you doing these using OpenCV? Since ZGE has built-in components ( Blur / Convolution / Expression ) for such effects*, or you could use a shader of course.Rado1 wrote:an initial set of image operations/filters: color transformation, flipping, Gaussian blur, and Canny edge detection
*Not sure whether BitmapLoad works properly when you've adjusted the size of a Bitmap using glTexImage2D though.
K
Re: Awsome
I'm using OpenCV filters, because they are applied to the captured images in runtime... and are relatively fast. How could be ZGE bitmap components used for captured bitmap which is set by glTexSubImage2D?Kjell wrote:Are you doing these using OpenCV? Since ZGE has built-in components ( Blur / Convolution / Expression ) for such effects*, or you could use a shader of course.
Yes, that's the question. Original bitmaps are 16x16, then scaled to 640x480 with my camera...Kjell wrote:*Not sure whether BitmapLoad works properly when you've adjusted the size of a Bitmap using glTexImage2D though.
Re: Awsome
Hi Rado1,
Just tried it .. looks like this using BitmapBlur
Obviously this wouldn't be a problem if the Bitmap component had integer values for width & height ( instead of the drop-down menus ). And using shaders it works just fine of course.
K
Just tried it .. looks like this using BitmapBlur
Obviously this wouldn't be a problem if the Bitmap component had integer values for width & height ( instead of the drop-down menus ). And using shaders it works just fine of course.
K
Re: Awsome
Sure, shaders would probably be the best solution for displaying images. However, the OpenCV image filters can also be used to prepare input images/videos for more complex image processing, such as edge detection, object recognition, motion recognition, etc. So they are usable...Kjell wrote:And using shaders it works just fine of course.
some tips for face detection
I've used OpenCV face detection/tracking. It is easy to do. Most examples of face tracking, that i have seen, is just face detection on every frame.2. I could try, for instance, object detection or motion analysis. StevenM do you have some experience with these topics? Could you please give me some good tutorials to be able to understand it fast and correctly?
Just note :
The cascade. Cascades are .xml files that contain data needed by the detector to detect a specific object and the files can be found in the ?:\opencv\sources\data folder.
These files would need to be copied to a specific folder with the zge DLL in order to implement face detection. There are also some other cascades on the web.
Detecting geometric shapes, blobs, lines does not require a cascade:
https://opencv-code.com/tutorials/detec ... -an-image/
2.prep image for face detection. You can use the image as is and that is what I normally do - but in order to improve speed and accuracy - scale down hi-res image. applying filters can improve detection and prevent false positives.
3.face_cascade.detectMultiScale( frame_gray, faces, 1.1, 2, 0|CV_HAAR_SCALE_IMAGE, Size(30, 30) );
parameters can be found on this page:
http://docs.opencv.org/modules/objdetec ... ation.html
faces might be tricky - it is a the variable that holds the face detection results array vec4(rectangular upper and lower corners) . To overlay a rectangle on the display, the rectangle need to be scaled back up if the image was scaled down for face detection.
the value 2(minimum neighbors) here has caused me to get a lot of false positives. raising this value can reduce false positives, but too high will result in non-detection and a performance hit.
CV_HAAR_SCALE_IMAGE - This constant in indicates: scale the image instead of the detector, arguably better than the default method.
Size = smallest (Width,Height) in pixels of a detected face.
I use python, but c++ code for face detection is here:
minimal example:
http://answers.opencv.org/question/2006 ... cognition/
opencv doc example:
http://docs.opencv.org/doc/tutorials/ob ... ifier.html
Re: Awsome
Hi guys,
K
Ah, i see .. makes sense.Rado1 wrote:However, the OpenCV image filters can also be used to prepare input images/videos for more complex image processing.
Alright, never mind thenVilleK wrote:I wouldn't recommend using ZGE's built-in filters though, they are not really designed for realtime use.
K
ZgeCV contains now also a simple object detection based on Haar Feature-based Cascade Classifier. After updating the dll you can run the attached demo project to detect frontal face and eyes. The quality of detection is not 100% and highly depends on the provided cascade classifier files. The attached project uses examples provided by OpenCV distribution.
Next plans:
1. Experiment with hand detection. I saw some interesting results on web able to detect hand and finger positions. I'll try to use it as a controller for ZGE games.
2. Porting the library to Android.
Next plans:
1. Experiment with hand detection. I saw some interesting results on web able to detect hand and finger positions. I'll try to use it as a controller for ZGE games.
2. Porting the library to Android.
- Attachments
-
- screenshot from face detection demo
- scr2.jpg (35.58 KiB) Viewed 31575 times
-
- demo3.zip
- face detection project file
- (146.69 KiB) Downloaded 894 times
Thank you
Yes! This is fantastic - downloading now.
I've seen this too. I think this is done using a combination of detection and tracking, but not sure.Experiment with hand detection. I saw some interesting results on web able to detect hand and finger positions.
This looks great. I can see it being used for interesting Visualizer effect experiments among other things. Hope I get a chance to use it.
Also clever that you include family members in your testing, perhaps they'll have more understanding why you spend much time in front of the computer
edit: just tried it and it works fine here.
Also clever that you include family members in your testing, perhaps they'll have more understanding why you spend much time in front of the computer
edit: just tried it and it works fine here.
I'm glad you like it. BTW I tested OpenCV 2.4.10 and it produces 4 times smaller dll than 3.0.0 beta; I achieved 3.17MB with the same functionality.
However, I'm still not satisfied. The problem is that both versions of OpenCV have memory leaks in very basic functions such as cvtColor(). Running of such a program results in many page faults and memory consumption is increasing. StevenM do you have the same experience?
However, I'm still not satisfied. The problem is that both versions of OpenCV have memory leaks in very basic functions such as cvtColor(). Running of such a program results in many page faults and memory consumption is increasing. StevenM do you have the same experience?