New stereoscopic cameras on Mars


JackDesBwa|3D
 

As you may know, the robot Perseverance has landed on Mars, confirmed by NASA’s JPL on Februabry 18, 2021 at 3:55 p.m. EST (12:55 p.m. PST / 8:55 p.m. UTC). It has several stereoscopic cameras onboard.

Fun fact: the famous first photo (https://apod.nasa.gov/apod/ap210219.html) comes from its front left hazard avoidance (stereoscopic) camera.

When I write this message, only the front left A and rear right photos of the hazard avoidance (stereoscopic) cameras returned to Earth (and in low 320×240 definition, and with protective cover) so that we cannot assemble a stereoscopic image yet.

I am not an expert of Perseverance, but I saw several stereoscopic cameras:
- 3 HazCams (2 front, 1 rear) b&w fisheye pointing to the ground
front_hazcams.jpg rear_hazcams.jpg
- 1 Navcam color with 42 centimeters base
navcams.jpg
- 1 Mastcam-Z color with zoom (on Curiosity, the equivalent instrument has mismatched fixed focal)
mastcam-z.jpg
There are a bunch of monoscopic cameras too.
Although there are high resolution color (and video) cameras, the bandwidth to Earth is limited so that the images we get are reduced versions of the captured data.

They are not the first stereoscopic cameras on Mars. About 1 year and half ago, I previewed a few hundreds images taken by Curiosity and found one pair particularly interesting. I processed them with difficulty but the result [hyperstereo with 42cm base] is interesting: https://stereopix.net/photo:koUNdPvaoA/?album:6q8dg The next image in the album is also a photo from Curiosity, but the hyper base with close scene make it harder to fuse.

JackDesBwa


John Rupkalvis
 

The stereo base on the Mars Perseverence hyperstereo images  can be post-reduced using software such as the Pretend Stereoid.  (NOTE:  that is 
Stereoid with two "e"s, NOT any relationship to the Steroid software, spelled with just a single "e".)

John A. Rupkalvis
stereoscope3d@...

Picture


On Fri, Feb 19, 2021 at 4:11 AM Stereopix Net <contact@...> wrote:
As you may know, the robot Perseverance has landed on Mars, confirmed by NASA’s JPL on Februabry 18, 2021 at 3:55 p.m. EST (12:55 p.m. PST / 8:55 p.m. UTC). It has several stereoscopic cameras onboard.

Fun fact: the famous first photo (https://apod.nasa.gov/apod/ap210219.html) comes from its front left hazard avoidance (stereoscopic) camera.

When I write this message, only the front left A and rear right photos of the hazard avoidance (stereoscopic) cameras returned to Earth (and in low 320×240 definition, and with protective cover) so that we cannot assemble a stereoscopic image yet.

I am not an expert of Perseverance, but I saw several stereoscopic cameras:
- 3 HazCams (2 front, 1 rear) b&w fisheye pointing to the ground
front_hazcams.jpg rear_hazcams.jpg
- 1 Navcam color with 42 centimeters base
navcams.jpg
- 1 Mastcam-Z color with zoom (on Curiosity, the equivalent instrument has mismatched fixed focal)
mastcam-z.jpg
There are a bunch of monoscopic cameras too.
Although there are high resolution color (and video) cameras, the bandwidth to Earth is limited so that the images we get are reduced versions of the captured data.

They are not the first stereoscopic cameras on Mars. About 1 year and half ago, I previewed a few hundreds images taken by Curiosity and found one pair particularly interesting. I processed them with difficulty but the result [hyperstereo with 42cm base] is interesting: https://stereopix.net/photo:koUNdPvaoA/?album:6q8dg The next image in the album is also a photo from Curiosity, but the hyper base with close scene make it harder to fuse.

JackDesBwa


JackDesBwa|3D
 

- 3 HazCams (2 front, 1 rear) b&w fisheye pointing to the ground


JackDesBwa


allen KC2KLC
 

Some years back I spent a fair amount of time poring through the Curiosity and Opportunity images. I found several stereo pairs that yielded nice results. I don't have a Stereopix account - but if anyone's interested in these pairs I would consider creating one. (Is there any issue with posting images from NASA without attribution? It would be impossible for me to fully identify the original images from which I generated my stereo pairs.)


Etienne Monneret (Perso)
 

Le 20/02/2021 à 00:10, John Rupkalvis a écrit :
The stereo base on the Mars Perseverence hyperstereo images  can be post-reduced using software such as the Pretend Stereoid.  (NOTE:  that is 
Stereoid with two "e"s, NOT any relationship to the Steroid software, spelled with just a single "e".)

Not sure this software still exists.



forum@trivision3d.com
 

Le 20/02/2021 à 00:15, Stereopix Net a écrit :
- 3 HazCams (2 front, 1 rear) b&w fisheye pointing to the ground


JackDesBwa
_._,_._,_

JMH/trivision3d : It's so colorful that it's not very colorful that it could be mistaken.The "red" planet isn't really red, it's time to review the coloring of the computer generated images on Mars (see example in the attachment "stereoscopisation" = stereo extraction of a 2D video traveling isue from 3D computer graphics).  To say that there are people ready to go there even if no return is possible.... you talk about a game of fun. The former French president François Hollande volunteered (joke) Hi hi. 
 Otherwise the sky is not black like on the Moon, so there is a certain atmosphere even if half as much as on Earth and the flying drone of Perseverance has rotors turning twice as fast to subdue itself.



JackDesBwa|3D
 

Some years back I spent a fair amount of time poring through the Curiosity and Opportunity images. I found several stereo pairs that yielded nice results.

I found a way to efficiently match the pairs yesterday, and looked at 5700 stereo shots of Curiosity. ^^
Most of them are of low interest from a graphical point of view, but there are indeed nice photogenic shots.
 
I don't have a Stereopix account - but if anyone's interested in these pairs I would consider creating one. (Is there any issue with posting images from NASA without attribution? It would be impossible for me to fully identify the original images from which I generated my stereo pairs.)

You should be right. There is a description zone to put credit for example.
By the way, if you think they are worth sharing, you can ask for an account on Stereopix.

Oh, it seems that I was wrong, they are color ones

The cameras were b&w in Curiosity and color in Perseverance.
I must have found the data for the wrong robot in my initial searches.

JackDesBwa


John Clement
 

The Mars atmosphere has only 1% the pressure as on Earth so there will be little Rayleigh scattering to produce color.  As to colorization, cameras do not see things the way our eyes do.  The brain compensates for lighting and tends to reproduce the color of objects independently of illumination.  Only the difference in color across boundaries is actually used to judge color.  This is called “color constancy”.  Intelligent software can correct badly faded photos to restore the color   we would have seen before fading.  Kodak digital ROC does an amazing job.  It doesn’t work well with in between colors, but some manual tweaking can improve it.  So were the photos run through such software, or are they the raw uncorrected versions?  Are the cameras just B&W or do they have color sensors, or both like our eyes.

 

John M. Clement

 

From: main@Photo-3d.groups.io <main@Photo-3d.groups.io> On Behalf Of forum@...
Sent: Saturday, February 20, 2021 5:53 AM
To: main@Photo-3d.groups.io
Subject: Re: [Photo-3d] New stereoscopic cameras on Mars

 

Le 20/02/2021 à 00:15, Stereopix Net a écrit :

- 3 HazCams (2 front, 1 rear) b&w fisheye pointing to the ground

 

 

JackDesBwa


allen KC2KLC
 

There is a detailed article on how Perseverance's Mastcam-Z camera captures color at https://mastcamz.asu.edu/the-mastcam-z-filter-set-how-perseverance-will-see-the-colors-of-mars/

allen


Etienne Monneret (Perso)
 

Le 23/02/2021 à 02:35, allen KC2KLC a écrit :
There is a detailed article on how Perseverance's Mastcam-Z camera captures color at https://mastcamz.asu.edu/the-mastcam-z-filter-set-how-perseverance-will-see-the-colors-of-mars/ <https://mastcamz.asu.edu/the-mastcam-z-filter-set-how-perseverance-will-see-the-colors-of-mars/>
Perseverance is wearing anaglyph glasses!

:D


JackDesBwa|3D
 

There is a detailed article on how Perseverance's Mastcam-Z camera captures color at https://mastcamz.asu.edu/the-mastcam-z-filter-set-how-perseverance-will-see-the-colors-of-mars/

Interesting article.
In terms of stereo, we learn that there are two filters that are matched (visible band and 800nm infrared)

Perseverance is wearing anaglyph glasses!

In the meaning that it can have different color per eye, yes, but they cannot be cyan/red.
Actually the visible colors filters (red, green, orange, red) are on the left eye, and infrared colors are on the right eye:
fig9_augmented.png
[Figure 9 of the article, augmented with visible spectrum color bar and sun spectrum (before being filtered by martian atmosphere)]

JackDesBwa


Gordon Au
 

FYI, I've had time in the last few days to find and edit some of stereo images they’ve released, and I put a bunch on my website: WorldofDepth.com (see the most recent posts). I'll add more tonight since they just released a bunch more from the Mastcam-Z stereocamera today.

- Gordon


JackDesBwa|3D
 

FYI, I've had time in the last few days to find and edit some of stereo images they’ve released, and I put a bunch on my website

Nice work Gordon!

On my side, I made a tool that I wanted to build for a long time: a website to match the stereo images of the rovers.
They are unprocessed files, but they are all there: https://mars.stereopix.net/ [those from Perseverance & Curiosity]
It can be used to select interesting shots for further processing (align, set window & co) or just previewing them forexample. The photos from Perseverance need more work to align them than the photos from Curiosity, but can be previewed as is nonetheless. Their metadata are also less good, so there might be more "false negative" matches on the website.

JackDesBwa


KenK
 

On Thu, Feb 25, 2021 at 04:48 AM, Stereopix Net wrote:
They are unprocessed files, but they are all there: https://mars.stereopix.net/ [those from Perseverance & Curiosity]
JackDesBwa -Your special mars site is awesome! Thanks

Gordon - You are faster than the ASU team! Their images are "coming soon". Their site is looking for "public submissions" but even that part is "coming soon".

Here is the ASU link. It is a flashy website but everything seems to be listed as "coming soon"
https://mastcamz.asu.edu/mars-images/

KenK


Gordon Au
 

On Thu, Feb 25, 2021 at 05:48 AM, Stereopix Net wrote:
On my side, I made a tool that I wanted to build for a long time: a website to match the stereo images of the rovers.
They are unprocessed files, but they are all there: https://mars.stereopix.net/
Wow, great tool, Jack! Thank you for sharing that, and for the compliments as well. I actually found some shots I'd been looking for there—images which I assumed were unpaired after I couldn't find their matches on in the NASA archive. Of course they may have been uploaded since last I checked.

A small suggestion: it would be useful for your tool to display not only the thumbnail links, but links to the image pages on the NASA site as well.

- Gordon


JackDesBwa|3D
 

A small suggestion: it would be useful for your tool to display not only the thumbnail links, but links to the image pages on the NASA site as well.

Not sure to understand.
When you select a pair, it displays the assembled image and writes links to the full size images at the bottom.

For an unknown reason (probably something related to how the hosting service stops and resumes the apps), it was rolled back to a previous version.
This version had a bug that may affect those links. Seems to be okay again.

JackDesBwa


Antonio F.G.
 

On Thu, Feb 25, 2021 at 04:48 AM, Stereopix Net wrote:
On my side, I made a tool that I wanted to build for a long time: a website to match the stereo images of the rovers.
They are unprocessed files, but they are all there: https://mars.stereopix.net/ [those from Perseverance & Curiosity]
It can be used to select interesting shots for further processing (align, set window & co) or just previewing them forexample.
This is an amazing work! Thanks a lot.
How have sorted the labyrinthine raw images of https://mars.nasa.gov/mars2020/multimedia/raw-images/ ?
I have tried to find matching pairs directly from the NASA site. It is apparently well organized: You can select the Mastcam-Z - Left and Right, which look like the ones with interesting landscapes. But the photos are not sorted by their time of shooting, which would make easy finding stereo pairs.


The photos from Perseverance need more work to align them than the photos from Curiosity, but can be previewed as is nonetheless. Their metadata are also less good, so there might be more "false negative" matches on the website.
What metadata? There is none in the images downloaded from the NASA site. Neither is in the download links provided in your site.

I have been trying to understand the relation between the shooting time given by NASA, the file name, and the date quoted in your site (below the download links).
To this end I have just compared one image of the Perseverance Left Mastcam-Z camera between Stereopix and Nasa. It was really difficult to find it in Nasa because of the chaotic ordering, but I finally found it:
In Stereopix site:
ZLF_0003_0667218749_000FDR_N0010052AUT_04096_034085J01.png   Date: 2021-02-24T19:37:41Z

In Nasa site:
ZLF_0003_0667218749_000FDR_N0010052AUT_04096_034085J01.png   Date: Feb. 24, 2021 (Sol 3) at the local mean solar time of 15:45:04.

It is quite confusing. Is there any relation between names and the two dates?


Trying to align these images I noticed a shortcoming of stmani3: it can not open png's. Something for my task list:-)

Regards
    Antonio


Gordon Au
 
Edited

On Fri, Feb 26, 2021 at 06:14 PM, Stereopix Net wrote:
A small suggestion: it would be useful for your tool to display not only the thumbnail links, but links to the image pages on the NASA site as well.
 
Not sure to understand.
When you select a pair, it displays the assembled image and writes links to the full size images at the bottom.
 

Jack, I mean links to the pages like this, which NASA has for each image, and which sometimes have additional useful information: http://mars.nasa.gov/mars2020/multimedia/raw-images/ZRF_0003_0667220795_000FDR_N0010052AUT_04096_034085J01 …and I think I kind of just answered my own question, in a way, haha. Apparently you can just take the original PNG file name without the extension, and append it to mars.nasa.gov…/raw-images, and that gives you the page in question. For convenience, it would still be nice for your tool to display these links beside the direct image links.

So Antonio, at least that's a way of finding the original NASA page given the filename. Yes, if their website sorted not only by Sol but by capture time, it would be so much more logical and useful…

- Gordon


Antonio F.G.
 

On Sat, Feb 27, 2021 at 12:53 PM, Gordon Au wrote:
... links to the pages like this, which NASA has for each image, and which sometimes have additional useful information: http://mars.nasa.gov/mars2020/multimedia/raw-images/ZRF_0003_0667220795_000FDR_N0010052AUT_04096_034085J01

Does anybody know how to get the focal length used in each of these images? For the Mastcam-Z in can range from 28 to 100mm.

It is important to know the focal length to make some alignment tricks.

Regards
    Antonio


JackDesBwa|3D
 

How have sorted the labyrinthine raw images ?

The script downloads a JSON feed from the site, put all the images of the day in one array (when it has to load multiples JSON), and loops in the list trying every pair of elements once [with anticipated stop when a match is found].
The condition of the match depends on the robot. With Curiosity, it is when the "date_taken" fields are the same and the instrument (except for LEFT/RIGHT) is the same [https://github.com/stereopix/mars-stereo/blob/master/resources/script.js#L223-L226]. With Perseverance, the date is probably wrong so I use a part of the imageid instead which seems to work, with in addition the name of the filter applied (which necessary for the mastcam) [https://github.com/stereopix/mars-stereo/blob/master/resources/script.js#L328-L333]

What metadata? There is none in the images downloaded from the NASA site. Neither is in the download links provided in your site.

The data in the JSON file are richer than what is displayed in the sites. I do not understand them all.

It is quite confusing. Is there any relation between names and the two dates?

There are two dates because they are expressed for different timezones.
One is Earth hour in UTC, the other is the time on Mars probably where is the rover.
The relation is not straightforward, but you can have it with some trigonometric relations. [https://www.giss.nasa.gov/tools/mars24/help/algorithm.html ; https://www.giss.nasa.gov/tools/mars24/help/notes.html]

I did not find the name convention for the files (only found it for Opportunity/Spirit)

For convenience, it would still be nice for your tool to display these links beside the direct image links.

I did not see these pages. I will add the link when I can.

Does anybody know how to get the focal length used in each of these images?

I do not know exactly, but I can give you a hint to orient your searches: CAHVOR
It seems (I do not know it) to be the name of the model used to calibrate the camera. From the matrix, and motivation, you can probably extract the focal length or other derived values you need.
Here is how the metadata on this part look like: (for the example image you gave before: ZLF_0003_0667218749_000FDR_N0010052AUT_04096_034085J01)
"camera": {
        "filter_name": "ZCAM_L0_RGB",
        "camera_vector": "(0.9034065773191104,0.15454228367729939,0.39996654686657906)",
        "camera_model_component_list": "(-1314.62,-155.946,4529.58);(0.905381,0.166472,0.390626);(-4.075e-05,-0.461466,0.489901);(0.877915,0.445148,-1.96715);(0.905032,0.166003,0.391633);(-20.4717,4758.45,267.611)",
        "camera_position": "(0.877915,0.445148,-1.96715)",
        "instrument": "MCZ_LEFT",
        "camera_model_type": "CAHVOR"
}


JackDesBwa