stillleben.diff module

Differentiation package for stillleben

Contents

Author: Arul Periyasamy <arul.periyasamy@ais.uni-bonn.de>

Functions

def compute_image_space_gradients(scene: Scene, render_result: RenderPassResult)
Gradient of intensity w.r.t. 2D pixel positions.
def backpropagate_gradient_to_poses(scene: Scene, render_result: RenderPassResult, grad_objective_wrt_rnd_img: torch.Tensor, visualize_grad = False)
Performs backpropagation of a gradient on the output image back to the object poses used to render the scene.
def apply_pose_delta(pose: torch.Tensor, delta: torch.Tensor, orthonormalize = True)
Applies a pose delta in the form of (\alpha,\beta,\gamma,a,b,c) to a 4x4 pose matrix.

Function documentation

def stillleben.diff.compute_image_space_gradients(scene: Scene, render_result: RenderPassResult)

Gradient of intensity w.r.t. 2D pixel positions.

Parameters
scene stillleben scene
render_result Render result from RenderPass
Returns (grad_x, grad_y, valid), where grad_x and grad_y are gradients w.r.t. X and Y pixel position for each pixel (shape HxWxC).

Basically, this answers the question of how the image will change when we move pixels in 2D.

def stillleben.diff.backpropagate_gradient_to_poses(scene: Scene, render_result: RenderPassResult, grad_objective_wrt_rnd_img: torch.Tensor, visualize_grad = False)

Performs backpropagation of a gradient on the output image back to the object poses used to render the scene.

Parameters
scene Scene
render_result Rendered frame
grad_objective_wrt_rnd_img 3xHxW float tensor with gradient of the objective w.r.t. the image.
visualize_grad Display grad visualization using visdom
Returns Nx6 float gradient of the objective w.r.t. the N poses.

Note that the orientation part of each pose is locally linearized, i.e.

T(\alpha,\beta,\gamma,a,b,c) = T_0 \left( \begin{matrix} 1 & -\gamma & \beta & a \\ \gamma & 1 & -\alpha & b \\ -\beta & \alpha & 1 & c \\ 0 & 0 & 0 & 1 \\ \end{matrix} \right)

def stillleben.diff.apply_pose_delta(pose: torch.Tensor, delta: torch.Tensor, orthonormalize = True)

Applies a pose delta in the form of (\alpha,\beta,\gamma,a,b,c) to a 4x4 pose matrix.

Parameters
pose 4x4 pose matrix. May also be batched (Bx4x4).
delta 6-dim delta vector. May also be batched (Bx6).
orthonormalize If true (default), perform an SVD for orthonormalization of the rotation matrix after the update.
Returns New 4x4 pose matrix. If the inputs are batched, this one is as well.

See backpropagate_gradient_to_poses() for the definition of delta.