Home

Geometry

Distributed under the terms of the CC BY-NC-ND 4.0 License.

  1. Points, Vectors and Normals
  2. Coordinate Systems
  3. Math Operations on Points and Vectors
  4. Matrices
  5. How Does Matrix Work: Part 1
  6. How Does Matrix Work: Part 2
  7. Transforming Points and Vectors
  8. Row Major vs Column Major Vector
  9. Matrix Operations
  10. Spherical Coordinates and Trigonometric Functions
  11. Creating an Orientation Matrix or Local Coordinate System
  12. Transforming Normals
  13. Source Code (external link GitHub)

Row Major vs Column Major Vector

Reading time: 17 mins.

Earlier in this lesson, we explained that vectors (or points) could be represented as [1x3] matrices (one row, three columns) or as [3x1] matrices (three rows, one column). Both forms are technically valid, and the choice between them is merely a matter of convention.

In the [1x3] matrix representation, the vector or point is in row-major order, meaning it is written as a row of three numbers. Conversely, in the [3x1] matrix representation, points or vectors are in column-major order, with the three coordinates of the vector or point written vertically as a column.

We use matrix representation for points and vectors to facilitate multiplication by [3x3] transformation matrices. For simplicity, we focus on [3x3] matrices rather than [4x4] matrices. Matrix multiplication is possible only when the number of columns in the left matrix matches the number of rows in the right matrix. Hence, a [1x3] matrix can be multiplied by a [3x3] matrix, but a [3x1] matrix cannot directly multiply a [3x3] matrix. This principle is demonstrated below, where the inner dimensions that match allow for valid multiplication (highlighted in green), resulting in a transformed point in the form of a [1x3] matrix:

$$ [1 \times \textcolor{green}{3}] \times [\textcolor{green}{3} \times 3] = \begin{bmatrix} x & y & z \end{bmatrix} \times \begin{bmatrix} c_{00} & c_{01} & c_{02} \\ c_{10} & c_{11} & c_{12} \\ c_{20} & c_{21} & c_{22} \end{bmatrix} = \begin{bmatrix} x' & y' & z' \end{bmatrix} $$

However, when the inner dimensions do not match (highlighted in red), multiplication is not feasible:

$$ [3 \times \textcolor{red}{1}]*[\textcolor{red}{3} \times 3] \rightarrow \begin{bmatrix} x\\ y\\z \end{bmatrix} * \begin{bmatrix} c_{00}&c_{01}&{c_{02}}\\ c_{10}&c_{11}&{c_{12}}\\ c_{20}&c_{21}&{c_{22}} \end{bmatrix} $$

To resolve this, instead of trying to multiply the vector or point by the matrix, we multiply the matrix by the vector, placing the point or vector to the right in the multiplication:

$$ [3 \times \textcolor{green}{3}] \times [\textcolor{green}{3} \times 1] = \begin{bmatrix} c_{00} & c_{01} & c_{02} \\ c_{10} & c_{11} & c_{12} \\ c_{20} & c_{21} & c_{22} \end{bmatrix} \times \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} x' \\ y' \\ z' \end{bmatrix} $$

This approach aligns with the conventions of matrix multiplication, ensuring that vectors or points can be correctly transformed by [3x3] matrices.

Note that the result of this operation is a transformed point written in the form of a [3x1] matrix. This means we start with a point and end with a transformed point, effectively solving our problem. To summarize, when we choose to represent vectors or points in row-major order ([1x3]), we position the point on the left side of the multiplication, with the [3x3] matrix on the right side within the multiplication sign. In mathematics, this arrangement is referred to as left or pre-multiplication. Conversely, if we opt to express vectors in column-major order ([3x1]), the [3x3] matrix is placed on the left side of the multiplication, with the vector or point on the right side, known as right or post-multiplication.

It's important to use these terms accurately. For example, Maya documentation states, "the matrices are post-multiplied in Maya. To transform a point P from object-space to world-space (P'), you would need to post-multiply by the worldMatrix. (P' = P x WM)", which might seem confusing because it actually describes a pre-multiplication scenario. However, the documentation is referring to the matrix's position relative to the point in this specific context, leading to a misuse of terminology. Correctly, it should state that in Maya, points and vectors are expressed as row-major vectors and are pre-multiplied, indicating the point or vector precedes the matrix in the multiplication order.

The differences between the two conventions can be summarized as follows, where P stands for Point, V for Vector, and M for Matrix.

Row-Major Order

Column-Major Order

Key observations include:

These conventions determine how transformations are applied to points or vectors in various graphics and mathematics applications, ensuring the correct application of operations like translation, rotation, and scaling.

Difference Between Row-Major and Column-Major Conventions

Understanding the difference between row-major and column-major conventions is not just a matter of notation but also has practical implications in mathematics and computer graphics. Here's an exploration of how these conventions affect matrix multiplication and transformations, and why one might be preferred over the other in certain contexts.

Row-Major vs. Column-Major in Matrix Multiplication

When calculating the product of two matrices, A and B, the process involves multiplying each element of a row in matrix A by the corresponding element of a column in matrix B and summing the results. The approach varies between the row-major and column-major conventions:

Despite the different approaches, both conventions should ideally yield the same result for a vector or point transformation. However, direct application shows differing outcomes due to the inherent mathematical structure of each convention. To align results, the matrix used in column-major multiplication requires transposition: To match outcomes, the column-major matrix must be transposed (for a refresher on matrix transposition, refer to the chapter on Matrix Operations):

$$\begin{bmatrix} a & d & g \\ b & e & h \\ c & f & i \end{bmatrix} * \begin{bmatrix}x\\y\\z\end{bmatrix}$$

Producing:

$${\begin{array}{l}x' = a * x + d * y + g * z\\y' = b * x + e * y + h * z\\z' = c * x + f * y + i * z\end{array}}$$

Ensuring that, regardless of the convention, the transformation results remain consistent.

Practical Application and Preference

The choice between row-major and column-major impacts the sequence of transformations. For instance:

The distinction between row-major and column-major conventions prompts a discussion on preference. Both approaches are legitimate, with mathematics and physics typically leaning towards column vectors for various reasons, including historical context and the practical explanations provided below. Scratchapixel adopts the row-major order convention, primarily because it aligns more intuitively with the concept of "transforming this point by this matrix" rather than the inverse.

Educational and API Considerations

In conclusion, while the row-major convention may offer simplicity and directness in teaching and certain applications, the column-major convention aligns closely with mathematical principles, making it the choice for many 3D graphics APIs. Understanding both conventions allows for flexibility in approaching problems in computer graphics and mathematics, ensuring the correct application of transformations regardless of the chosen convention.

Implication in Coding: Does it Impact Performance?

Choosing between row-major and column-major order in coding does not just concern the conventions of writing. It's also about the practicality concerning the computer's architecture and how it processes data. When dealing with [4x4] matrices in programming, particularly in C++, a matrix might typically be implemented as follows:

class Matrix44 { 
    float m[4][4]; 
    // Additional class details
};

In this setup, the 16 coefficients of a [4x4] matrix are stored in a contiguous two-dimensional array. This arrangement means they're laid out in memory in a row-major fashion: c00, c01, c02, c03, ..., c33.

Row-Major Order Access Pattern:

In row-major order vector-matrix multiplication, the elements are accessed non-sequentially:

// Row-major order
x' = x * c00 + y * c10 + z * c20;
y' = x * c01 + y * c11 + z * c21;
z' = x * c02 + y * c12 + z * c22;

This pattern means that for calculating x', elements are accessed with gaps, leading to potential cache misses due to non-sequential memory access, affecting CPU performance.

Column-Major Order Access Pattern:

Conversely, in column-major order, the elements are accessed sequentially:

// Column-major order
x' = c00 * x + c01 * y + c02 * z;
y' = c10 * x + c11 * y + c12 * z;
z' = c20 * x + c21 * y + c22 * z;

Sequential access aligns with the CPU's cache mechanism, reducing cache misses and potentially enhancing performance.

Theoretically, column-major order could offer better performance due to improved cache usage. However, in practice, especially with compiler optimizations (-O, -O2, -O3 flags), the difference might not be significant. Modern compilers are capable of optimizing memory access patterns in multi-dimensional arrays, mitigating the performance gap between row-major and column-major access patterns.

The provided C++ example showcases a matrix and vector class template, illustrating how vector-matrix multiplication can be implemented for both row-major and column-major conventions. The performance test, involving a large number of iterations, aims to measure the execution time difference between these conventions. Yet, it's noted that real-world performance differences might not be significant due to compiler optimizations.

In summary, while column-major order might theoretically offer performance advantages due to sequential memory access patterns, the actual impact on performance may vary depending on specific use cases and compiler optimizations.

template<typename T> 
class Vec3 
{ 
public: 
    Vec3(T xx, T yy, T zz) : x(xx), y(yy), z(zz) {} 
    T x, y, z, w; 
}; 
 
template<typename T> 
class Matrix44 
{ 
public: 
    T m[4][4]; 
    Vec3<T> multVecMatrix(const Vec3<T> &v) 
    { 
#ifdef ROWMAJOR 
        return Vec3<T>( 
            v.x * m[0][0] + v.y * m[1][0] + v.z * m[2][0], 
            v.x * m[0][1] + v.y * m[1][1] + v.z * m[2][1], 
            v.x * m[0][2] + v.y * m[1][2] + v.z * m[2][2]); 
#else 
        return Vec3<T>( 
            v.x * m[0][0] + v.y * m[0][1] + v.z * m[0][2], 
            v.x * m[1][0] + v.y * m[1][1] + v.z * m[1][2], 
            v.x * m[2][0] + v.y * m[2][1] + v.z * m[2][2]); 
#endif 
    } 
}; 
 
#include <cmath> 
#include <cstdlib> 
#include <cstdio> 
#include <ctime> 
 
#define MAX_ITER 10e8 
 
int main(int argc, char **argv) 
{ 
    clock_t start = clock(); 
    Vec3<float> v(1, 2, 3); 
    Matrix44<float> M; 
    float *tmp = &M.m[0][0]; 
    for (int i = 0; i < 16; i++) *(tmp + i) = drand48(); 
    for (int i = 0; i < MAX_ITER; ++i) { 
        Vec3<float> vt = M.multVecMatrix(v); 
    } 
    fprintf(stderr, "Clock time %f\n", (clock() - start) / float(CLOCKS_PER_SEC)); 
    return 0; 
} 

In computing, the concepts of row-major and column-major order describe the memory layout for multidimensional arrays. This distinction is crucial for understanding data access patterns, which can affect performance, especially in operations like matrix multiplication.

Row-major Order

Column-major Order

Understanding these layouts is essential for optimizing memory access. In C/C++, we typically deal with row-major order, but it's important to be aware of column-major order, especially when interfacing with software or libraries that use a different convention, such as OpenGL.

The question of whether real-time APIs like OpenGL use row-major or column-major matrix notation in their documentation, and how these matrices are organized in memory, has historically been a source of confusion. While modern APIs, apart from DirectX (leaving examples like Vulkan), predominantly utilize column-order matrices, they also frequently offer the flexibility within shaders to specify the order in which matrix coefficients should be arranged in memory. This level of adaptability essentially grants all the versatility one might need in matrix manipulation. Though as a consequence, even the most experienced programmers may find themselves experimenting with the order of vector and matrix multiplications several times until the visual output aligns with expectations. Sigh!

It's essential to distinguish between the representation of matrix coefficients on paper and their arrangement in memory. The terms "row-major" or "column-major" order can refer to either the notation used in mathematical documentation, impacting the interpretation and function of the coefficients, or the manner in which these coefficients are organized within memory. For instance, in a 16-element float array representing a 4x4 matrix, the first four elements m[0], m[1], m[2], m[3] could correspond to a, b, c, d or to a, e, i, m, where each letter represents a specific coefficient in the matrix:

$$\begin{bmatrix} a & b & c & d \\ e & f & g & h \\ i & j & k & l \\ m & n & o & p \end{bmatrix}$$

Summary

Differences Between Row-Major and Column-Major Vector Conventions in Mathematics:

A reader posted a question on Stackoverflow suggesting the topic of row-major versus column-major matrix conventions is confusing. Here's another attempt at shedding light on the matter, distinguishing between mathematical conventions and implementation details in programming.

Mathematics: Row vs. Column Vector Convention

  • In mathematics, vectors can be represented in two ways: as row or column vectors.

    • Row vectors are written as a 1x4 matrix (e.g., \(vM\) where \(v\) is the row vector and \(M\) is a 4x4 matrix), adhering to the row-major convention.

    • Column vectors are noted vertically, fitting the column-major convention, thus multiplying a matrix by a vector is expressed as \(Mv\).

    • The distinction is based on the vector's position relative to the matrix, leading to terms like "left or pre-multiplication" for row vectors and "right or post-multiplication" for column vectors.

  • Transformation order is vital. For instance, to apply transformations in sequence (translate, then rotate, then scale), the order of operations differs between conventions:

    • Column-major: \(v' = S * R * T * v\)

    • Row-major: \(v' = v * T * R * S\)

Computer Implementation: Row vs. Column-Major Memory Layout

  • Implementing these concepts in C++ offers flexibility in how matrix coefficients are stored in memory, independent of mathematical conventions. This is the row/column-major memory layout issue.

  • For example, a matrix class in C++ might store coefficients in a contiguous float array. The layout choice doesn't dictate the mathematical convention used but should align with the chosen convention for correct vector-matrix and matrix-matrix operations.

  • It's crucial to map matrix coefficients in memory thoughtfully to ensure correct access during operations. Different storage strategies might be chosen, each requiring adapted access patterns in code but not affecting the mathematical convention applied.

Practical Example in C++

Consider the implementation of a vector-matrix multiplication function. Regardless of the convention (row or column vector), the multiplication's outcome should be consistent. The function needs to multiply the vector's coordinates (\(x, y, z\)) with the correct matrix coefficients, highlighting the importance of understanding how these coefficients are stored in memory:

Vector3 vecMatMult(
    Vector3 v,
    float AXx, float AXy, float AXz, float Tx,
    float AYx, float AYy, float AYz, float Ty,
    float AZz, float AZy, float AZz, float Tz) 
{
    return Vector3(
        v.x * AXx + v.y * AYx + v.z * AZx + Tx,
        v.x * AXy + v.y * AYy + v.z * AZy + Ty,
        v.x * AXz + v.y * AZz + v.z * AZz + Tz);
}

This function illustrates that the physical storage of matrix coefficients (row or column-major layout) and the mathematical convention used (row or column vectors) are separate concerns. The key is to apply the correct access pattern when performing operations.

Conclusion

The distinction between row-major and column-major conventions involves both mathematical notation and implementation details. While the choice of convention affects the order of operations and notation in mathematical expressions, the implementation in programming languages like C++ offers flexibility in how these conventions are applied. Understanding both aspects is crucial for correctly implementing mathematical operations in software.

previousnext

Found a problem with this page?

Want to fix the problem yourself? Learn how to contribute!

Source this file on GitHub

Report a problem with this content on GitHub