August 2, 2011

kkrzewniak kkrzewniak
Lab Rat
218 posts

[solved] Using the camera api on the n950

Page  
1

Hi was anyone successful in using the qml camera api on the n950.
I’m using the following code:

  1. import QtQuick 1.0
  2. import QtMultimediaKit 1.1
  3.  
  4. Item {
  5.  
  6.     Camera{
  7.         id: camera_item
  8.         anchors.fill: parent
  9.  
  10.         focus: true
  11.  
  12.         flashMode: Camera.FlashRedEyeReduction
  13.         whiteBalanceMode: Camera.WhiteBalanceFlash
  14.         exposureCompensation: -1.0
  15.  
  16.         onCaptureFailed: console.log("Failed: " + message)
  17.         onStateChanged: console.log("Camera state: " + state)
  18.         onImageCaptured: {
  19.             console.log("Preview url: " + preview)
  20.            //process the image
  21.         }
  22.         onError: console.log("Error: " + errorString)
  23.     }
  24.  
  25.     MouseArea{
  26.         anchors.fill: parent
  27.         onClicked: {
  28.             console.log("Error: " + camera_item.errorString)
  29.             console.log("Taking image")
  30.             console.log("Unloaded State - " + Camera.UnloadedState)
  31.             console.log("Loaded State - " + Camera.LoadedState)
  32.             console.log("Active State - " + Camera.ActiveState)
  33.             console.log("Current camera state: " + camera_item.cameraState)
  34.             camera_item.captureImage()
  35.         }
  36.     }
  37.  
  38.     Component.onCompleted: camera_item.start()
  39. }

I get the following output and the viewfinder is never shown :|

  1. Error:
  2. Taking image
  3. Unloaded State - 0
  4. Loaded State - 1
  5. Active State - 2
  6. Current camera state: 2
  7. Failed: Camera not ready

Thanks

 Signature 

Me, Grimlock, not “nice dino”. ME BASH BRAINS!

18 replies

August 2, 2011

cpscotti cpscotti
Lab Rat
33 posts

Got the same result here.
(exactly same result, n950 too).

 Signature 

cpscotti.com/blog/ - Used when I need to kill some time at work :D

August 2, 2011

kkrzewniak kkrzewniak
Lab Rat
218 posts

now The question is: Am I missing something, is the code invalid? Or should we consider filing a bug report. :)

 Signature 

Me, Grimlock, not “nice dino”. ME BASH BRAINS!

August 2, 2011

cpscotti cpscotti
Lab Rat
33 posts

I tried looking into http://doc.qt.nokia.com/qtmobility-1.2/qml-camera.html which (afaik) is the official doc to see if we were missing something to no avail.

Soo, We should either fill a bug report to the API or to the docs..

 Signature 

cpscotti.com/blog/ - Used when I need to kill some time at work :D

August 4, 2011

dmytro.poplavskiy dmytro.popla..
Lab Rat
10 posts

I tried this example on device using
qmlviewer -opengl -fullscreen cameratest.qml
and it worked fine. I ssh to user account.

This issue is very likely caused by aegis security system, it’s necessary to add file like this

  1. <aegis>
  2.     <request>
  3.         <credential name="GRP::video" />
  4.         <credential name="GRP::pulse-access" />
  5.         <for path="absolute path to application" />
  6.     </request>
  7. </aegis>

to debian/appname.aegis
and rebuild/reinstall deb package

August 4, 2011

kkrzewniak kkrzewniak
Lab Rat
218 posts

Thank you very much dmytro I figured it was some kind of policy missing and was experimenting with aegis :)
but never found the credential names I needed to include.

Is there perhaps a document describing the credentials needed for certain features.
Thanks

 Signature 

Me, Grimlock, not “nice dino”. ME BASH BRAINS!

August 4, 2011

cpscotti cpscotti
Lab Rat
33 posts

Thanks dmytro!
just one thing, the example works now but it segfaults when I swipe it to the background. Do you guys see this too?
I reckon it might be an “expected” behavior since a proper app should put the camera on stand by when inactive..
Any thoughts on this?

 Signature 

cpscotti.com/blog/ - Used when I need to kill some time at work :D

August 5, 2011

kkrzewniak kkrzewniak
Lab Rat
218 posts

Yeah have the same behavior here but I’m 99% sure this will be solved once I put the camera on stand by (it’s required behavior afaik, imagine having your app running in the background with the camera active, preventing other apps form using it).

 Signature 

Me, Grimlock, not “nice dino”. ME BASH BRAINS!

August 5, 2011

cpscotti cpscotti
Lab Rat
33 posts

Yep, me too BUT then we have another problem.
How do you put the camera in stand by?
I tried adding:

  1.     Connections {
  2.         target: platformWindow
  3.         onActiveChanged: {
  4.             if (platformWindow.active) {
  5.                 console.log("Active")
  6.                 camera_item.start()
  7.             } else {
  8.                 console.log("Inactive")
  9.                 camera_item.stop()
  10.             }
  11.         }
  12.     }

But it doesn’t work at all.

This is the output I get when I swipe it to the background:

  1. ...
  2. Meego graphics system destroyed
  3. Inactive
  4. ResourceSet::proceedIfImFirst()...allowing only request directly.
  5. ResourceSet::update().... updating...
  6. ResourceEngine(1)::updateResources() - **************** locking....
  7. Converted Resource 0x03 to 0x08
  8. Converted Resource 0x0a to 0x800
  9. Converted Resource 0x0b to 0x1000
  10. All resources as bitmask is 0x1808
  11. ResourceEngine(1) - update 1:5
  12. ResourceSet::proceedIfImFirst()...queuing request 2.
  13. ResourceSet::proceedIfImFirst()...queuing request:Acquire.
  14. Illegal instruction (core dumped)

The weird thing is that Camera.start()/Camera.stop() doesn’t seem to do anything at all.
Actually, if you remove all your calls to start(), the camera is still loaded normally.
Cool isn’t it?

 Signature 

cpscotti.com/blog/ - Used when I need to kill some time at work :D

August 8, 2011

matrixx matrixx
Lab Rat
50 posts

The reason why aegis credentials are not documented properly yet, is that the final set of credentials hasn’t been frozen yet. Proper documentation of which credentials are needed for which functionality, including a complete list of all available credentials will be published later.

kkrzewniak wrote:
Thank you very much dmytro I figured it was some kind of policy missing and was experimenting with aegis :)
but never found the credential names I needed to include.

Is there perhaps a document describing the credentials needed for certain features.
Thanks

 Signature 

Nokia Certified Qt Specialist

August 10, 2011

cpscotti cpscotti
Lab Rat
33 posts

Thanks matrixx for the clarification on the creds.

Do you (or anyone else) have any idea on how to STOP the camera? (so it doesn’t crash when windows is inactive)..

 Signature 

cpscotti.com/blog/ - Used when I need to kill some time at work :D

August 10, 2011

kkrzewniak kkrzewniak
Lab Rat
218 posts

calling stop would suffice the only problem is that

  1. platformWindow
  2. onActiveChanged

Is called after I get “Meego graphics system destroyed” and if I remember correctly if I stop the camera before that by hand it’s all ok

 Signature 

Me, Grimlock, not “nice dino”. ME BASH BRAINS!

August 11, 2011

dmytro.poplavskiy dmytro.popla..
Lab Rat
10 posts

You can try to use the raster graphics system for application with QGLWidget based viewport to ensure camera doesn’t loose gl context.

The more correct solution would be to stop() or even better unload() the camera when minimized, as suggested before.

I’m also concerned to see “Meego graphics system destroyed” before “Inactive”, it’s better to unload the camera before graphics system is destroyed.

August 24, 2011

deion deion
Lab Rat
69 posts

Hi Dmytro,

I am also trying to use QCamera APIs on N950.
My first issue was on making the aegis manifest file but this is now resolved thanks to the community help.

Now I am having other issues:
1) I am trying to grab frames from the camera. I understand (and you provided this solution before) that for this I need to override QAbstractVideoSurface, re-implement supportedPixelFormats() and present(); I have done that and on my custom QAbstractVideoSurface the method is called with the video frames (QVideoFrame param).

How I used it :

  1.     camera = new QCamera;
  2.     videoSurface = new VideoSurface(); // VideoSurface extends QAbstractVideoSurface
  3.     camera->setViewfinder(videoSurface);
  4.     camera->setCaptureMode(QCamera::CaptureVideo);
  5.     camera->start();

As I said this works (present is called with desired QVideoFrame s). However during format negotiation in VideoSurface::supportedPixelFormats() the only pixel format available is QVideoFrame::Format_UYVY. This means that in VideoSurface::present I can not use the provided QVideoFrame param to construct a QImage from the raw frame data and using this QImage display the frames on the screen (viewfinder). Should I implement a custom color space conversion from UYVY to RGB to be able to display the frames? I saw in another post of yours that you suggested using this code to inject the custom video surface (http://developer.qt.nokia.com/forums/viewthread/370):

  1. QVideoRendererControl *control = camera->service()->requestControl<QVideoRendererControl>();
  2. if (control) control->setSurface(yourSurface);

(and I suppose that instead of

  1. camera->setViewfinder(videoSurface);

use

  1. QCameraViewfinder* viewFinder = new QCameraViewfinder();
  2. camera->setViewFinder(viewFinder);
  3. viewFinder->show();

which will correctly display a viewfinder)

BUT, in this line

  1. QVideoRendererControl* control = camera->service()->requestControl<QVideoRendererControl>();

the returned control is NULL

So how can I grab QCamera video frames and also present a viewfinder on the screen? The only solution is through colorspace conversion? This is time consuming …

2) Another issue is how to set the desired video resolution? I used a QMediaRecoder and set the desired settings on this object to force desired video resolution on QCamera(even though I am not interested in capturing video on a file, I am only interested of grabbing video frames at certain resolution and FPS):

  1.     camera = new QCamera;
  2.     videoSurface = new VideoSurface(); // VideoSurface extends QAbstractVideoSurface
  3.     camera->setViewfinder(videoSurface);
  4.     camera->setCaptureMode(QCamera::CaptureVideo);
  5.  
  6.     mediaRecorder = new QMediaRecorder(camera);
  7.     QVideoEncoderSettings videoSettings = mediaRecorder->videoSettings();
  8.     videoSettings.setResolution(QSize(320, 240));
  9.     mediaRecorder->setEncodingSettings(mediaRecorder->audioSettings(),videoSettings);
  10.  
  11.     camera->start();

Thanks & regards,
Ionut Dediu

August 25, 2011

dmytro.poplavskiy dmytro.popla..
Lab Rat
10 posts

Hi Ionut,

you spotted two areas I’d like to improve in next versions on QCamera API:
to have access to video frames simultaneously with efficient rendering (this is also related by lesser extent to QMediaPlayer) and viewfinder resolution settings (and probably framerate, not so sure about this).

deion wrote:
Hi Dmytro,

I am also trying to use QCamera APIs on N950.
My first issue was on making the aegis manifest file but this is now resolved thanks to the community help.

Now I am having other issues:
1) I am trying to grab frames from the camera. I understand (and you provided this solution before) that for this I need to override QAbstractVideoSurface, re-implement supportedPixelFormats() and present(); I have done that and on my custom QAbstractVideoSurface the method is called with the video frames (QVideoFrame param).

How I used it :

  1.     camera = new QCamera;
  2.     videoSurface = new VideoSurface(); // VideoSurface extends QAbstractVideoSurface
  3.     camera->setViewfinder(videoSurface);
  4.     camera->setCaptureMode(QCamera::CaptureVideo);
  5.     camera->start();

As I said this works (present is called with desired QVideoFrame s). However during format negotiation in VideoSurface::supportedPixelFormats() the only pixel format available is QVideoFrame::Format_UYVY. This means that in VideoSurface::present I can not use the provided QVideoFrame param to construct a QImage from the raw frame data and using this QImage display the frames on the screen (viewfinder). Should I implement a custom color space conversion from UYVY to RGB to be able to display the frames?

Colorspace transformation is disabled in camerabin pipeline by default for performance reasons, it’s possible to enable it with

QCameraControl *control = camera->service()->requestControl<QCameraControl>();
control->setProperty(“viewfinderColorSpaceConversion”, true);

This flag is gstreamer backend specific (it should be documented).
You may also try to report a limited supported set of RGB formats, gst colorspace convertor on N950 can convert only to a few RGB formats efficiently.

You can also have a look at qgraphicsvideoitem_maemo5.cpp for a very fast UYVY to RGB16 transformation, if license is suitable for you.

I’m thinking about video frames probe like API to have read only access to video frames in parallel with efficient rendering, and video filter like API to modify video frame before rendering, but I can’t promise when/if it’s done.
Any feedback/ideas in this area are very welcome.


I saw in another post of yours that you suggested using this code to inject the custom video surface (http://developer.qt.nokia.com/forums/viewthread/370):

  1. QVideoRendererControl *control = camera->service()->requestControl<QVideoRendererControl>();
  2. if (control) control->setSurface(yourSurface);

(and I suppose that instead of

  1. camera->setViewfinder(videoSurface);

use

  1. QCameraViewfinder* viewFinder = new QCameraViewfinder();
  2. camera->setViewFinder(viewFinder);
  3. viewFinder->show();

which will correctly display a viewfinder)

BUT, in this line

  1. QVideoRendererControl* control = camera->service()->requestControl<QVideoRendererControl>();

the returned control is NULL

Regarding “camera->setViewfinder(videoSurface);” vs request control, those methods should work the same if only one output is used (setViewfinder() detaches previous output) and camera->setViewfinder() is simpler and more preferred.

QMediaService API allows backend to have multiple video outputs attached, but current gstreamer backend doesn’t support this, that’s why requestControl<QVideoRendererControl>() returns NULL.

So how can I grab QCamera video frames and also present a viewfinder on the screen? The only solution is through colorspace conversion? This is time consuming …

2) Another issue is how to set the desired video resolution? I used a QMediaRecoder and set the desired settings on this object to force desired video resolution on QCamera(even though I am not interested in capturing video on a file, I am only interested of grabbing video frames at certain resolution and FPS):

  1.     camera = new QCamera;
  2.     videoSurface = new VideoSurface(); // VideoSurface extends QAbstractVideoSurface
  3.     camera->setViewfinder(videoSurface);
  4.     camera->setCaptureMode(QCamera::CaptureVideo);
  5.  
  6.     mediaRecorder = new QMediaRecorder(camera);
  7.     QVideoEncoderSettings videoSettings = mediaRecorder->videoSettings();
  8.     videoSettings.setResolution(QSize(320, 240));
  9.     mediaRecorder->setEncodingSettings(mediaRecorder->audioSettings(),videoSettings);
  10.  
  11.     camera->start();

Unfortunately the current QCamera API doesn’t provide a viewfinder resolution control,
using QMediaRecorder is a valid workaround. It should not have any negative performance impact.

August 26, 2011

deion deion
Lab Rat
69 posts

Hi Dmytro,

Thanks a lot for your answers, they were really helpful.
I wanted to post a longer reply to highlight more issues regarding the differences between Symbian and Meego regarding the camera capabilities; I will do this later, maybe it will be useful to you to get more feedback on the usage scenarios for the QCamera APIs.
Right now I wanted to ask you about the method

  1. bool QAbstractVideoSurface::present ( const QVideoFrame & frame )

What is the lifetime of the frame param? I can not cache it and use it later after returning from the present() method, is this right?
If so, how can I safely make a copy of the frame param into another QVideoFrame object so that I can use it later? I’m thinking of adding the frames into a queue/list and later process these separately from another thread. How can I copy a QVideoFrame?

Thanks & regards,
Ionut

Page  
1

  ‹‹ Compiling kqoauth for symbian devices in windows      [Solved]tab size ››

You must log in to post a reply. Not a member yet? Register here!