Row Major vs Column Major Vector
Reading time: 17 mins.Earlier in this lesson, we explained that vectors (or points) could be represented as [1x3] matrices (one row, three columns) or as [3x1] matrices (three rows, one column). Both forms are technically valid, and the choice between them is merely a matter of convention.

Vector represented as a [1x3] matrix:
$$V = \begin{bmatrix} x & y & z \end{bmatrix}$$ 
Vector represented as a [3x1] matrix:
$$V = \begin{bmatrix} x \\ y \\ z \end{bmatrix}$$
In the [1x3] matrix representation, the vector or point is in rowmajor order, meaning it is written as a row of three numbers. Conversely, in the [3x1] matrix representation, points or vectors are in columnmajor order, with the three coordinates of the vector or point written vertically as a column.
We use matrix representation for points and vectors to facilitate multiplication by [3x3] transformation matrices. For simplicity, we focus on [3x3] matrices rather than [4x4] matrices. Matrix multiplication is possible only when the number of columns in the left matrix matches the number of rows in the right matrix. Hence, a [1x3] matrix can be multiplied by a [3x3] matrix, but a [3x1] matrix cannot directly multiply a [3x3] matrix. This principle is demonstrated below, where the inner dimensions that match allow for valid multiplication (highlighted in green), resulting in a transformed point in the form of a [1x3] matrix:
$$ [1 \times \textcolor{green}{3}] \times [\textcolor{green}{3} \times 3] = \begin{bmatrix} x & y & z \end{bmatrix} \times \begin{bmatrix} c_{00} & c_{01} & c_{02} \\ c_{10} & c_{11} & c_{12} \\ c_{20} & c_{21} & c_{22} \end{bmatrix} = \begin{bmatrix} x' & y' & z' \end{bmatrix} $$However, when the inner dimensions do not match (highlighted in red), multiplication is not feasible:
$$ [3 \times \textcolor{red}{1}]*[\textcolor{red}{3} \times 3] \rightarrow \begin{bmatrix} x\\ y\\z \end{bmatrix} * \begin{bmatrix} c_{00}&c_{01}&{c_{02}}\\ c_{10}&c_{11}&{c_{12}}\\ c_{20}&c_{21}&{c_{22}} \end{bmatrix} $$To resolve this, instead of trying to multiply the vector or point by the matrix, we multiply the matrix by the vector, placing the point or vector to the right in the multiplication:
$$ [3 \times \textcolor{green}{3}] \times [\textcolor{green}{3} \times 1] = \begin{bmatrix} c_{00} & c_{01} & c_{02} \\ c_{10} & c_{11} & c_{12} \\ c_{20} & c_{21} & c_{22} \end{bmatrix} \times \begin{bmatrix} x \\ y \\ z \end{bmatrix} = \begin{bmatrix} x' \\ y' \\ z' \end{bmatrix} $$This approach aligns with the conventions of matrix multiplication, ensuring that vectors or points can be correctly transformed by [3x3] matrices.
Note that the result of this operation is a transformed point written in the form of a [3x1] matrix. This means we start with a point and end with a transformed point, effectively solving our problem. To summarize, when we choose to represent vectors or points in rowmajor order ([1x3]), we position the point on the left side of the multiplication, with the [3x3] matrix on the right side within the multiplication sign. In mathematics, this arrangement is referred to as left or premultiplication. Conversely, if we opt to express vectors in columnmajor order ([3x1]), the [3x3] matrix is placed on the left side of the multiplication, with the vector or point on the right side, known as right or postmultiplication.
It's important to use these terms accurately. For example, Maya documentation states, "the matrices are postmultiplied in Maya. To transform a point P from objectspace to worldspace (P'), you would need to postmultiply by the worldMatrix. (P' = P x WM)", which might seem confusing because it actually describes a premultiplication scenario. However, the documentation is referring to the matrix's position relative to the point in this specific context, leading to a misuse of terminology. Correctly, it should state that in Maya, points and vectors are expressed as rowmajor vectors and are premultiplied, indicating the point or vector precedes the matrix in the multiplication order.
The differences between the two conventions can be summarized as follows, where P stands for Point, V for Vector, and M for Matrix.
RowMajor Order

Representation: Points or vectors (P/V) are expressed as [1x3] matrices, such as \(P/V = \begin{bmatrix}x & y & z\end{bmatrix}\).

Multiplication Convention: Left or premultiplication. The point or vector is placed on the left side of the multiplication operation.

Operation: \(P/V * M\). The point or vector is multiplied by the matrix, indicating that the transformation matrix follows the vector in the operation.
ColumnMajor Order

Representation: Points or vectors (P/V) are expressed as [3x1] matrices, such as
$$P/V = \begin{bmatrix}x \\ y \\ z\end{bmatrix}$$ 
Multiplication Convention: Right or postmultiplication. The matrix is placed on the left side of the multiplication operation.

Operation: \(M * P/V\). The matrix multiplies the point or vector, indicating that the vector or point follows the transformation matrix in the operation.
Key observations include:

In rowmajor order, vectors or points are typically represented in a single row (as [1x3] matrices), and multiplication by a transformation matrix (M) is done from the right (P/V * M), known as premultiplication.

In columnmajor order, vectors or points are represented in a single column (as [3x1] matrices), and the transformation matrix multiplies from the left (M * P/V), referred to as postmultiplication.
These conventions determine how transformations are applied to points or vectors in various graphics and mathematics applications, ensuring the correct application of operations like translation, rotation, and scaling.
Difference Between RowMajor and ColumnMajor Conventions
Understanding the difference between rowmajor and columnmajor conventions is not just a matter of notation but also has practical implications in mathematics and computer graphics. Here's an exploration of how these conventions affect matrix multiplication and transformations, and why one might be preferred over the other in certain contexts.
RowMajor vs. ColumnMajor in Matrix Multiplication
When calculating the product of two matrices, A and B, the process involves multiplying each element of a row in matrix A by the corresponding element of a column in matrix B and summing the results. The approach varies between the rowmajor and columnmajor conventions:

RowMajor Order: Multiplication is performed as follows:
$$\begin{bmatrix}x & y & z\end{bmatrix} * \begin{bmatrix} a & b & c \\ d & e & f \\ g & h & i \end{bmatrix}$$Resulting in:
$${\begin{array}{l}x' = x * a + y * d + z * g\\y' = x * b + y * e + z * h\\z' = x * c + y * f + z * i\end{array}}$$ 
ColumnMajor Order: The multiplication is:
$$\begin{bmatrix} a & b & c \\ d & e & f \\ g & h & i \end{bmatrix} * \begin{bmatrix}x\\y\\z\end{bmatrix}$$Producing:
$${\begin{array}{l}x' = a * x + b * y + c * z\\y' = d * x + e * y + f * z\\z' = g * x + h * y + i * z\end{array} }$$
Despite the different approaches, both conventions should ideally yield the same result for a vector or point transformation. However, direct application shows differing outcomes due to the inherent mathematical structure of each convention. To align results, the matrix used in columnmajor multiplication requires transposition: To match outcomes, the columnmajor matrix must be transposed (for a refresher on matrix transposition, refer to the chapter on Matrix Operations):
$$\begin{bmatrix} a & d & g \\ b & e & h \\ c & f & i \end{bmatrix} * \begin{bmatrix}x\\y\\z\end{bmatrix}$$Producing:
$${\begin{array}{l}x' = a * x + d * y + g * z\\y' = b * x + e * y + h * z\\z' = c * x + f * y + i * z\end{array}}$$Ensuring that, regardless of the convention, the transformation results remain consistent.
Practical Application and Preference
The choice between rowmajor and columnmajor impacts the sequence of transformations. For instance:

RowMajor Transformations: Sequential order is preserved, such as:
$$P'=P * T * R_z * R_y$$ 
ColumnMajor Transformations: Requires reverse order, which might seem counterintuitive:
$$P'=R_y * R_z * T * P$$
The distinction between rowmajor and columnmajor conventions prompts a discussion on preference. Both approaches are legitimate, with mathematics and physics typically leaning towards column vectors for various reasons, including historical context and the practical explanations provided below. Scratchapixel adopts the rowmajor order convention, primarily because it aligns more intuitively with the concept of "transforming this point by this matrix" rather than the inverse.
Educational and API Considerations

RowMajor Convention: Favored for its intuitive sequence of transformations, making it easier to teach. Used in graphics applications and APIs like Maya and DirectX.

ColumnMajor Convention: Used by OpenGL and other 3D APIs, aligning more closely with mathematical function evaluation and composition, suggesting a deeper reason for its preference in certain contexts.
In conclusion, while the rowmajor convention may offer simplicity and directness in teaching and certain applications, the columnmajor convention aligns closely with mathematical principles, making it the choice for many 3D graphics APIs. Understanding both conventions allows for flexibility in approaching problems in computer graphics and mathematics, ensuring the correct application of transformations regardless of the chosen convention.
Implication in Coding: Does it Impact Performance?
Choosing between rowmajor and columnmajor order in coding does not just concern the conventions of writing. It's also about the practicality concerning the computer's architecture and how it processes data. When dealing with [4x4] matrices in programming, particularly in C++, a matrix might typically be implemented as follows:
class Matrix44 { float m[4][4]; // Additional class details };
In this setup, the 16 coefficients of a [4x4] matrix are stored in a contiguous twodimensional array. This arrangement means they're laid out in memory in a rowmajor fashion: c00, c01, c02, c03, ..., c33
.
RowMajor Order Access Pattern:
In rowmajor order vectormatrix multiplication, the elements are accessed nonsequentially:
// Rowmajor order x' = x * c00 + y * c10 + z * c20; y' = x * c01 + y * c11 + z * c21; z' = x * c02 + y * c12 + z * c22;
This pattern means that for calculating x'
, elements are accessed with gaps, leading to potential cache misses due to nonsequential memory access, affecting CPU performance.
ColumnMajor Order Access Pattern:
Conversely, in columnmajor order, the elements are accessed sequentially:
// Columnmajor order x' = c00 * x + c01 * y + c02 * z; y' = c10 * x + c11 * y + c12 * z; z' = c20 * x + c21 * y + c22 * z;
Sequential access aligns with the CPU's cache mechanism, reducing cache misses and potentially enhancing performance.
Theoretically, columnmajor order could offer better performance due to improved cache usage. However, in practice, especially with compiler optimizations (O
, O2
, O3
flags), the difference might not be significant. Modern compilers are capable of optimizing memory access patterns in multidimensional arrays, mitigating the performance gap between rowmajor and columnmajor access patterns.
The provided C++ example showcases a matrix and vector class template, illustrating how vectormatrix multiplication can be implemented for both rowmajor and columnmajor conventions. The performance test, involving a large number of iterations, aims to measure the execution time difference between these conventions. Yet, it's noted that realworld performance differences might not be significant due to compiler optimizations.
In summary, while columnmajor order might theoretically offer performance advantages due to sequential memory access patterns, the actual impact on performance may vary depending on specific use cases and compiler optimizations.
template<typename T> class Vec3 { public: Vec3(T xx, T yy, T zz) : x(xx), y(yy), z(zz) {} T x, y, z, w; }; template<typename T> class Matrix44 { public: T m[4][4]; Vec3<T> multVecMatrix(const Vec3<T> &v) { #ifdef ROWMAJOR return Vec3<T>( v.x * m[0][0] + v.y * m[1][0] + v.z * m[2][0], v.x * m[0][1] + v.y * m[1][1] + v.z * m[2][1], v.x * m[0][2] + v.y * m[1][2] + v.z * m[2][2]); #else return Vec3<T>( v.x * m[0][0] + v.y * m[0][1] + v.z * m[0][2], v.x * m[1][0] + v.y * m[1][1] + v.z * m[1][2], v.x * m[2][0] + v.y * m[2][1] + v.z * m[2][2]); #endif } }; #include <cmath> #include <cstdlib> #include <cstdio> #include <ctime> #define MAX_ITER 10e8 int main(int argc, char **argv) { clock_t start = clock(); Vec3<float> v(1, 2, 3); Matrix44<float> M; float *tmp = &M.m[0][0]; for (int i = 0; i < 16; i++) *(tmp + i) = drand48(); for (int i = 0; i < MAX_ITER; ++i) { Vec3<float> vt = M.multVecMatrix(v); } fprintf(stderr, "Clock time %f\n", (clock()  start) / float(CLOCKS_PER_SEC)); return 0; }
In computing, the concepts of rowmajor and columnmajor order describe the memory layout for multidimensional arrays. This distinction is crucial for understanding data access patterns, which can affect performance, especially in operations like matrix multiplication.
Rowmajor Order

Used by languages like C and C++, where elements are stored sequentially from left to right, top to bottom.

For a matrix
\(M = \begin{bmatrix}1&2&3\\4&5&6\end{bmatrix}\),
it's represented in C/C++ as
float m[2][3] = {{1, 2, 3}, {4, 5, 6}};
, with elements laid out in memory as 1, 2, 3, 4, 5, 6.
Columnmajor Order

Adopted by languages such as FORTRAN and MATLAB, this approach stores elements from top to bottom, left to right.

Using the same matrix \(M\), elements would be stored in memory as 1, 4, 2, 5, 3, 6.
Understanding these layouts is essential for optimizing memory access. In C/C++, we typically deal with rowmajor order, but it's important to be aware of columnmajor order, especially when interfacing with software or libraries that use a different convention, such as OpenGL.
The question of whether realtime APIs like OpenGL use rowmajor or columnmajor matrix notation in their documentation, and how these matrices are organized in memory, has historically been a source of confusion. While modern APIs, apart from DirectX (leaving examples like Vulkan), predominantly utilize columnorder matrices, they also frequently offer the flexibility within shaders to specify the order in which matrix coefficients should be arranged in memory. This level of adaptability essentially grants all the versatility one might need in matrix manipulation. Though as a consequence, even the most experienced programmers may find themselves experimenting with the order of vector and matrix multiplications several times until the visual output aligns with expectations. Sigh!
It's essential to distinguish between the representation of matrix coefficients on paper and their arrangement in memory. The terms "rowmajor" or "columnmajor" order can refer to either the notation used in mathematical documentation, impacting the interpretation and function of the coefficients, or the manner in which these coefficients are organized within memory. For instance, in a 16element float array representing a 4x4 matrix, the first four elements m[0]
, m[1]
, m[2]
, m[3]
could correspond to a
, b
, c
, d
or to a
, e
, i
, m
, where each letter represents a specific coefficient in the matrix:
Summary
Differences Between RowMajor and ColumnMajor Vector Conventions in Mathematics:

Vector Representation:

RowMajor: \(P/V=\begin{bmatrix}x & y & z\end{bmatrix}\)

ColumnMajor: \(P/V=\begin{bmatrix}x \\ y \\ z\end{bmatrix}\)


Multiplication Type:

RowMajor: Premultiplication (vM).

ColumnMajor: Postmultiplication (Mv).


Transformation Order:

RowMajor: Sequential order is maintained (e.g., \(P'=P*T*R_z*R_y\))

ColumnMajor: Reverse order of transformation (e.g., \(P'=R_y*R_z*T*P\))


API Compatibility:

RowMajor: Compatible with Direct X, Maya.

ColumnMajor: Compatible with OpenGL, PBRT, Blender, Vulkan.


Basis (Axes) Representation in the Matrix:

RowMajor: Rows represent the coordinate system's bases (colorcoded: red for xaxis, green for yaxis, blue for zaxis).
$$ {\begin{bmatrix} \color{red}{c_{00}}& \color{red}{c_{01}}&\color{red}{c_{02}}&0\\ \color{green}{c_{10}}& \color{green}{c_{11}}&\color{green}{c_{12}}&0\\ \color{blue}{c_{20}}& \color{blue}{c_{21}}&\color{blue}{c_{22}}&0\\ 0&0&0&1 \end{bmatrix} } $$ 
ColumnMajor: Columns represent the coordinate system's bases (colorcoded: red for xaxis, green for yaxis, blue for zaxis).
$$ { \begin{bmatrix} \color{red}{c_{00}}& \color{green}{c_{01}}&\color{blue}{c_{02}}&0\\ \color{red}{c_{10}}& \color{green}{c_{11}}&\color{blue}{c_{12}}&0\\ \color{red}{c_{20}}& \color{green}{c_{21}}&\color{blue}{c_{22}}&0\\ 0&0&0&1 \end{bmatrix} } $$


Translation Matrices:

RowMajor:
$$ {\begin{bmatrix} 1&0&0&0\\ 0&1&0&0\\ 0&0&1&0\\ Tx&Ty&Tz&1 \end{bmatrix} } $$ 
ColumnMajor:
$$ {\begin{bmatrix} 1&0&0&Tx\\ 0&1&0&Ty\\ 0&0&1&Tz\\ 0&0&0&1 \end{bmatrix} } $$


Matrix Transposition for Convention Switch:

RowMajor: Transpose the matrix for use in columnmajor contexts

ColumnMajor: Transpose the matrix for use in rowmajor contexts

A reader posted a question on Stackoverflow suggesting the topic of rowmajor versus columnmajor matrix conventions is confusing. Here's another attempt at shedding light on the matter, distinguishing between mathematical conventions and implementation details in programming.
Mathematics: Row vs. Column Vector Convention

In mathematics, vectors can be represented in two ways: as row or column vectors.

Row vectors are written as a 1x4 matrix (e.g., \(vM\) where \(v\) is the row vector and \(M\) is a 4x4 matrix), adhering to the rowmajor convention.

Column vectors are noted vertically, fitting the columnmajor convention, thus multiplying a matrix by a vector is expressed as \(Mv\).

The distinction is based on the vector's position relative to the matrix, leading to terms like "left or premultiplication" for row vectors and "right or postmultiplication" for column vectors.


Transformation order is vital. For instance, to apply transformations in sequence (translate, then rotate, then scale), the order of operations differs between conventions:

Columnmajor: \(v' = S * R * T * v\)

Rowmajor: \(v' = v * T * R * S\)

Computer Implementation: Row vs. ColumnMajor Memory Layout

Implementing these concepts in C++ offers flexibility in how matrix coefficients are stored in memory, independent of mathematical conventions. This is the row/columnmajor memory layout issue.

For example, a matrix class in C++ might store coefficients in a contiguous float array. The layout choice doesn't dictate the mathematical convention used but should align with the chosen convention for correct vectormatrix and matrixmatrix operations.

It's crucial to map matrix coefficients in memory thoughtfully to ensure correct access during operations. Different storage strategies might be chosen, each requiring adapted access patterns in code but not affecting the mathematical convention applied.
Practical Example in C++
Consider the implementation of a vectormatrix multiplication function. Regardless of the convention (row or column vector), the multiplication's outcome should be consistent. The function needs to multiply the vector's coordinates (\(x, y, z\)) with the correct matrix coefficients, highlighting the importance of understanding how these coefficients are stored in memory:
Vector3 vecMatMult( Vector3 v, float AXx, float AXy, float AXz, float Tx, float AYx, float AYy, float AYz, float Ty, float AZz, float AZy, float AZz, float Tz) { return Vector3( v.x * AXx + v.y * AYx + v.z * AZx + Tx, v.x * AXy + v.y * AYy + v.z * AZy + Ty, v.x * AXz + v.y * AZz + v.z * AZz + Tz); }
This function illustrates that the physical storage of matrix coefficients (row or columnmajor layout) and the mathematical convention used (row or column vectors) are separate concerns. The key is to apply the correct access pattern when performing operations.
Conclusion
The distinction between rowmajor and columnmajor conventions involves both mathematical notation and implementation details. While the choice of convention affects the order of operations and notation in mathematical expressions, the implementation in programming languages like C++ offers flexibility in how these conventions are applied. Understanding both aspects is crucial for correctly implementing mathematical operations in software.