The origin of the coordinate system of pointclouds

[solmazhajmohammadi]

@ZongyangLi @pless
Right now the origin of the coordinate system is set to the location of calibration object during the first calibration run.

• We can change the origin in the configuration file and reprocess the data. (Requires Fraunhofer to change it)
• Or we can find the exact location by some test scans and define a new origin and add a transformation matrix to the metadata.
  We can go for the second option and @smarshall-bmr can do the scans to find the exact origin.

[ZongyangLi]

@solmazhajmohammadi Thanks! We do not have to change the origin. I just want to make sure:

1. Coordinate of ground surface is constant, it will not change as gantry position z changed;
2. A parameter of converting coordinate into field meters, as y scan distance is 21.8 meters, Y coordinate range in west scanner is from -600 to 21200, can we simply set this parameter to 1000, and assuming that Y coordinate -600 represent field position 0 in Y axis?

[dlebauer]

To follow up on a discussion from today's call, it does not seem that the scanner3DTop metadata includes information about the sensor position or field of view as is the case with all other sensors. It does, however, provide the location of the scanner box:

Here is the metadata from sites/ua-mac/raw_data/scanner3DTop/2016-08-12/2016-08-12__02-24-02-273/b6bc5837-3aaf-4443-8dad-289137e3fa9b_metadata.json:

{
  "lemnatec_measurement_metadata": {
    "user_given_metadata": {
      "experiment title": "Sorghum field experiment",
      "experiment responsible": "to be named",
      "project": "TERRA-REF",
      "instrument": "gantry at Maricopa phenotyping facility",
      "instrument": "field gantry",
      "location": "Maricopa phenotyping facility",
      "date of sowing": "2016-04-19",
      "date of emergence": "2016-04-25",
      "campaign": "seedling emergence and early vigor",
      "mission or scan": "3D Scan Field 0.33MperS"
    },
    "gantry_system_fixed_metadata": {
      "system manufacturer": "LemnaTec Corp.",
      "gantry fixed data 1": "Todo",
      "gantry fixed data 2": "Todo"
    },
    "gantry_system_variable_metadata": {
      "time": "08/12/2016 02:24:02",
      "position x [m]": "48.7015",
      "position y [m]": "0",
      "position z [m]": "1.114",
      "speed x [m/s]": "0",
      "speed y [m/s]": "0",
      "speed z [m/s]": "0",
      "camera box light 1 is on": "False",
      "camera box light 2 is on": "False",
      "camera box light 3 is on": "False",
      "camera box light 4 is on": "False",
      "scanIsInPositiveDirection": "True",
      "scanDistance [m]": "21.8",
      "scanSpeed [m/s]": "0.33",
      "scanMode": "Triggered"
    },
    "sensor_fixed_metadata": {
      "sensor manufacturer": "Fraunhofer-Entwicklungszentrum R?ntgentechnik EZRT, Ber?hrungslose Mess- und Pr?fsysteme, www.iis.fraunhofer.de",
      "sensor product name": "Custom made 3D Scanner"
    },
    "sensor_variable_metadata": {
      "current setting Exposure [microS]": "70",
      "current setting Calculate 3D files": "0",
      "current setting Laser detection threshold": "512",
      "current setting Scanlines per output file": "100000",
      "current setting Scan direction (automatically set at runtime)": "1",
      "current setting Scan distance (automatically set at runtime) [mm]": "21800",
      "current setting Scan speed (automatically set at runtime) [microMeter/s]": "100000"
    }
  }
}

by contrast, the flir camera has a much more detailed sensor fixed metadata field:

    "sensor_fixed_metadata": {
      "sensor manufacturer": "FLIR Systems",
      "sensor product name": "A645",
      "sensor serial number": "13070601",
      "sensor description": "IR Sensor 640x480",
      "sensor purpose": "measure infrared radiation from soil/crop surface",
      "buildin variable optics": "",
      "location in gantry system": "camera box, facing ground",
      "location in camera box x [m]": "0.877",
      "location in camera box y [m]": "1.361",
      "location in camera box z [m]": "0.520",
      "field of view x [m]": "1.496",
      "field of view y [m]": "1.105",
      "output data format": "16bit Grey",
      "calibration available": "calibration parameters from FLIR to convert gray value to temperature",
      "sensor computer interface": "GigE",
      "sensor id": "ir camera box"
    },

[JeffWhiteAZ]

David,
Thanks. It looks like we need to review the metadata for completeness and correctness for each instrument. Stuart, Maria and I looked at the 3D scanner metadata, and I was surprised that there was no data for location within the sensor box. Sensor position in the box is precisely the sort of data that might get changed if we ever modify the sensor box to include new instruments.
Do we have an official metadata expert who cross checks all of the data?

[markus-radermacher-lemnatec]

Due to a glitch in the config file, only a few meta data infos got into the 3D results. This has been fixed now. Solmaz is working on the transformation from the 3D Ply files into the gantry/world coordinate system.

[solmazhajmohammadi]

@smarshall-bmr Do we have the 3D scans with checker boards to find the pointclouds origin and its transformation to the gantry coordinate system?
Probably it is better to do it when the plants are short, otherwise it would be hard and inaccurate.

[smarshall-bmr]

@solmazhajmohammadi I have to build a die to hold the checkerboards at exactly 90 degrees from each other, otherwise the z axis measurement isn't going to be too accurate. That and the field is quite soggy right now.

[solmazhajmohammadi]

@smarshall-bmr any update on this?
@ZongyangLi I think this is what you were referring to, as soon as I get the data, I can give you the transformation to gantry coordinate and origin of point cloud
@smarshall-bmr can you please let me know when you are doing the scan, we can check to make sure we have all the measurements to find the origin relative to the gantry origin

[ghost]

@solmazhajmohammadi - have you heard back from Stewart? After you talk to him can you please update this issue?

[smarshall-bmr]

@rachelshekar @solmazhajmohammadi The first die I built for holding the targets wasn't very accurate. I'm planning on trying again soon.

[ghost]

@smarshall-bmr can you please give an update on this?

[solmazhajmohammadi]

@smarshall-bmr It woud be fine If your target is not exactly 90degree. We can figure out the error using simple 2 level target later.

[solmazhajmohammadi]

@smarshall-bmr For this issue please run two full row scan in both directions. (Issue#265)

[ghost]

Changing this to April milestone since dependent on terraref/computing-pipeline#265

[ghost]

@max-zilla - please update

[max-zilla]

we will handle this in the cleaning piece - will discuss when i meet with @craig-willis and @jterstriep

[ghost]

@max-zilla please update this issue

[max-zilla]

@craig-willis we just need to apply the values Zongyang supplied above to the metadata coming from the 3dscanner I believe, in your metadata cleaning function for that sensor.

we can close this once done.

[craig-willis]

@max-zilla I'll need some clarification of where this gets applied. It looks like we can either put the offset values in the sensor fixed metadata and use them elsewhere in the pipeline, or have the metadata cleaning function to the necessary calculation and add the origin as properties.

[craig-willis]

@solmazhajmohammadi @max-zilla @ZongyangLi
Per discussions last week, I'm looking at adding the point cloud origin calculation to the new terrautils library given the information in #44 (comment).

I still have two questions:

1. How do I find the center of the 3D scanner (for X)?
2. The comment mentions that the origin is calculated based on the position of the west scanner, but we have no data for scanner3DWest (or scanner3DEast), only scanner3DTop? I do see some sparse data for scanner3DLowerOnWestSide and scanner3DLowerOnEastSide.

[craig-willis]

@max-zilla @ZongyangLi @solmazhajmohammadi
Please review the _calculatePointCloudOrigin function in terrautils. This is now using the standardized Lemnatec metadata. Please let me know if you have any question.

https://github.com/terraref/terrautils/blob/master/terrautils/metadata/lemnatec.py#L679-L708

[ZongyangLi]

@craig-willis @solmazhajmohammadi

For your two questions on Jul 23:

1. I have no idea either where is the center of the 3d scanner

2. In the scanner3dTop, there are two sets of file that one from west laser scanner and the other one comes from east laser scanner. they are all in the same directory.

3. If I apply +3450mm in positive scan and +25711mm in negative scan, and use the season 4's borders' data to extract a single plant, I got a 'half-half' plant visualization rather than an exactly single plant. examples:
   
   

I checked the field book this time and I am using plot borders rather than plot center.

I am not sure what makes this happened, but I think this may not be the case we would like to add into terrautils.

[ZongyangLi]

@craig-willis @solmazhajmohammadi
I noticed the update to the borders that different from that in my codes, so it fits better now. So don't worry about the 'half-half' plant visualization, it should be fine now.

[ghost]

@solmazhajmohammadi can this issue be closed?

[max-zilla]

@solmazhajmohammadi I integrated your code for the heightmap geoTIFF registration. images in negative scan direction looked correct, but positive direction was still off.

@craig-willis and I took a look yesterday, specifically at the formula to get GPS bounding box from metadata. we determined that we had 2 similar formulas in the code:

1. _calculatePointCloudOrigin() in terrautils/lemnatec, which craig implemented based on this post from this issue: The origin of the coordinate system of pointclouds #44 (comment)

https://github.com/terraref/terrautils/blob/master/terrautils/lemnatec.py#L729
(there may be a typo here)

this code adds a point_cloud_origin to the metadata that we upload to Clowder.

point_cloud_origin = {}
    if (not 'plc_control_not_available' in corrected_gantry_variable_md 
        and 'position_m' in corrected_gantry_variable_md
        and 'scan_direction_is_positive' in corrected_gantry_variable_md
        and 'scanner_west_location_in_camera_box_m' in fixed_md):
            
        point_cloud_origin["z"] =  float(corrected_gantry_variable_md['position_m']['z']) - 3.445
        point_cloud_origin["x"] =  float(fixed_md["scanner_west_location_in_camera_box_m"]["x"]) - 0.082
        if (corrected_gantry_variable_md["scan_direction_is_positive"] == "True"):
            point_cloud_origin["y"] = float(corrected_gantry_variable_md['position_m']['y']) + 3.450
        else:
            point_cloud_origin["y"] = float(corrected_gantry_variable_md['position_m']['z']) + 25.711

    return point_cloud_origin

2. the edits you made to terrautils/spatial.py which use the gantry & camera box positions:

if sensor=='scanner3DTop':
        # Swap X and Y because we rotate 90 degress
        fov_x = float(fov_y) if fov_y else 0
        scanInDistance = 21.8
        fov_y = scanInDistance #replace this with scanInDistance from gantry-metadata
		
		# this should go into the switch-case loop to choose scandirection and east/west scanner
		
        #Negative scan: West Scanner:
        center_position = ( float(gantry_x) + float(cambox_x) + 0.082,
            	                float(gantry_y) + float(2*cambox_y) - scanInDistance/2 - 4.263, #Might be less than this
                	            float(gantry_z) + float(cambox_z) )
                            
        #Negative scan: East Scanner:
        center_position = ( float(gantry_x) + float(cambox_x) + 0.082,
                            float(gantry_y) + float(2*cambox_y) - scanInDistance/2 - 0.046, 
                            float(gantry_z) + float(cambox_z) )
                            
        #Positive scan: West Scanner
        center_position = ( float(gantry_x) + float(cambox_x) + 0.082,
                            float(gantry_y) + float(2*cambox_y) + scanInDistance/2 + 3.23  ,
                            float(gantry_z) + float(cambox_z) )
                            
        #Positive scan: East Scanner
        center_position = ( float(gantry_x) + float(cambox_x) + 0.082,
                            float(gantry_y) + float(2*cambox_y) + scanInDistance/2 - 1.44 ,
                            float(gantry_z) + float(cambox_z) )

It seems to me these should be combined - we should fix the pointCloudOrigin() function with latest information, then make the spatial.py function just use those.

I implemented a first attempt to combine these formulas:
https://github.com/terraref/terrautils/blob/terrautils_final_prep_2/terrautils/lemnatec.py#L784
calculatePointCloudOrigin() now returns two coordinates, one for east and one for west. otherwise the formula is similar to what it was before, although i fixed a type where 'z' was used instead of 'y'.

https://github.com/terraref/terrautils/blob/terrautils_final_prep_2/terrautils/spatial.py#L100
the spatial.py code now attempts to build off of the PCO instead of gantry position. there was not a perfect match between your code here and what we have in the PCO. I kept some of your same coefficients and the scandistance/2 part.

I'll let you take a look at these things, but then I would like to update the scanner3d test repo we were using on Cloud9 with an example of both scan directions so we can get things working consistently. let me know if this doesn't make sense.

[max-zilla]

current example outputs:

the "East" halves of 2 scans. positive direction on top, negative on bottom.

same scans with West added. positive direction scan is too far west, but negative scan direction is pretty close.

[max-zilla]

@ZongyangLi and @solmazhajmohammadi are looking into using this to georeference the LAS files we create from merged PLY files.

[craig-willis]

Stale issue -- closing. Open a new issue if work remains.