Elliptic-curve Diffie-Hellman (ECDH) lets two parties agree on a shared secret using points on a curve instead of numbers on a clock, with keys a fraction of the size that finite-field methods need. In practice, almost every ECDH exchange on the internet uses one of two curves: P-256, standardised by the United States National Institute of Standards and Technology (NIST), or Curve25519, designed by Daniel Bernstein and used through a function called X25519. Both offer roughly the same strength against classical attackers, both are allowed in Transport Layer Security (TLS) version 1.3, and both produce 32-byte private keys. Yet they are built on different curve shapes, represent points differently, compute in different ways and, most importantly, demand different checks on the values a peer sends. This article opens up both: the field arithmetic underneath, how a point is added and doubled, why the Montgomery ladder matters, how keys are encoded on the wire, and the input rules that, if skipped, let an attacker extract a private key. It ends with how to choose between them and what that means for the artificial intelligence (AI) agents whose every connection starts with one of these exchanges.

Two curves, one job

An elliptic curve, for cryptography, is the set of points whose coordinates satisfy an equation of the form y squared equals x cubed plus a few terms, with all arithmetic done modulo a large prime. Points can be added with a geometric rule, and adding a point to itself many times, called scalar multiplication, is easy forward but hard to reverse. ECDH uses that asymmetry: each side keeps a secret count, publishes the point reached by that many additions from an agreed base point, and multiplies the other side's point by its own count to reach a shared point.

P-256 was published in the late 1990s and became part of NIST's Digital Signature Standard. It uses the short Weierstrass form, y squared equals x cubed minus 3x plus a fixed constant, over a 256-bit prime, and its group of points has prime size. Curve25519, published in 2006, uses the Montgomery form over the prime 2 to the 255th minus 19, which is where its name comes from. Its group size is 8 times a large prime, and its X25519 function, standardised in Request for Comments (RFC) 7748, works with only the x-coordinate of each point.

Both target about 128 bits of classical security. The differences are in engineering: how easy each is to implement in constant time, how much each trusts its input, and which compliance regimes have historically required which. Those differences are the subject of the rest of this article.

Both deliver ECDH at about 128-bit strength in TLS 1.3, but they differ in group structure, point format and the checks they demand. What the two curves share P-256prime orderx and y coordinatesvalidate every point Bothabout 128-bitTLS 1.3 key shares32-byte private keys X25519cofactor 8x-coordinate onlyclamping, any input
Both deliver ECDH at about 128-bit strength in TLS 1.3, but they differ in group structure, point format and the checks they demand.

The arithmetic underneath

Every curve operation is built from additions, multiplications and divisions modulo the field prime, and the prime is chosen so that the remainder step is fast. The Curve25519 prime, 2 to the 255th minus 19, sits just below a power of two, so wrapping a large product around it needs little more than a multiplication by 19 and an addition. The P-256 prime is a so-called generalised Mersenne prime, a sum and difference of powers of two that also allows fast reduction, though with more steps.

Point operations are where the curve shape shows up. On a toy curve, y squared equals x cubed plus 2x plus 2 modulo 17, take the base point (5, 1) and double it. The tangent line's slope is three times x squared plus 2, divided by twice y: 3 times 25 plus 2 is 77, which wraps to 9, and dividing by 2 means multiplying by 9, the number that times 2 gives 1 modulo 17, so the slope is 81, which wraps to 13. The new x is the slope squared minus twice the old x, 169 minus 10, which wraps to 6, and the new y follows as 3. Doubling (5, 1) gives (6, 3).

Notice that these formulas use the curve's coefficient in front of x, here 2, but never its constant term. That detail looks harmless and turns out to be the root of the invalid-curve attack explained below. Also notice the division: real implementations avoid slow divisions by working in projective coordinates, which carry an extra coordinate so that divisions can be postponed to a single one at the end. For prime-order Weierstrass curves such as P-256, so-called complete formulas exist that handle every case, including doubling and the point at infinity, with one code path, which helps avoid branches that leak timing.

Scalar multiplication and the Montgomery ladder

Computing k times a point for a 256-bit secret k uses the binary digits of k. The schoolbook method, double-and-add, doubles at every bit and adds only when the bit is 1. That makes the sequence of operations depend on the secret: an attacker who can measure timing, power use or electromagnetic emissions can read the bits off the pattern. Constant-time implementation means the sequence of operations and memory accesses must not depend on secret values.

Peter Montgomery's ladder solves this elegantly. It keeps two points, R0 and R1, with R1 always equal to R0 plus the base point. For every bit, it performs exactly one addition and one doubling; the bit only decides which of the two registers receives which result. Implementations do that with a conditional swap built from arithmetic masks rather than an if-statement, so the processor runs identical instructions for every bit.

X25519 was designed around the ladder. Montgomery-form curves allow a particularly cheap ladder step that needs only x-coordinates, which is why X25519 public keys are just an x value. P-256 implementations can also use ladders or fixed-window methods in constant time, but the design leaves more choices to the implementer and, historically, more room for mistakes.

Every bit runs the same swap, add, double and swap sequence, so timing reveals nothing about the secret. One ladder step per bit Read next bitof the secret scalar Conditional swapmasked, no branch Add and doublealways both Swap backsame mask next bit Same operations every bit, whatever the bit's value
Every bit runs the same swap, add, double and swap sequence, so timing reveals nothing about the secret.

Encodings on the wire

A public key has to travel as bytes, and the two curves encode it very differently. A P-256 point has two 32-byte coordinates. The uncompressed encoding defined in the Standards for Efficient Cryptography (SEC 1) is a single byte with value 4 followed by x and y, 65 bytes in total. The compressed encoding sends a byte with value 2 or 3, recording whether y is even or odd, followed by x alone, 33 bytes; the receiver recomputes y with a square root.

An X25519 public key is always exactly 32 bytes: the x-coordinate in little-endian order, meaning the least significant byte first. The top bit of the last byte is ignored on input. There is no flag byte and no y-coordinate to check, because the ladder never needs one.

Encodings are a common source of interoperability bugs and occasionally of security bugs. Byte order differs between the two curves; some formats wrap keys in structures such as SubjectPublicKeyInfo, the X.509 container that also names the algorithm; and code that accepts several encodings must validate each one. Mixing up which curve or which encoding a key uses can make a verifier silently compute on the wrong values.

How each curve's public and private keys are encoded.
ItemP-256X25519
Public key, full65 bytes: 4, x, y32 bytes: x, little-endian
Public key, compressed33 bytes: 2 or 3, xNot applicable
Private key32-byte integer below the group order32 random bytes, clamped when used
Coordinate checksx and y below p, point on curveNone needed; top bit ignored

Input rules for P-256: validate every point

Because P-256's addition formulas never use the curve's constant term, they will happily compute on a point that is not on P-256 at all. Such a point lies on some other curve with the same leading coefficients, and that other curve may have points of very small order. Return to the toy curve modulo 17: the point (1, 0) is not on it, since 1 plus 2 plus 2 is 5, not 0. But it does lie on the curve whose constant is 14 instead of 2, and there it has order 2. Feed it to the toy curve's arithmetic with a secret count k, and the result is (1, 0) when k is odd and the point at infinity when k is even. An attacker who can observe which happened learns one bit of k.

This is the invalid-curve attack, described by Ingrid Biehl, Bernd Meyer and Volker Müller in 2000. Against a server that reuses its ECDH private key, an attacker sends a series of such points of different small orders, learns the key modulo each, and combines the pieces. In 2015 Tibor Jager, Jörg Schwenk and Juraj Somorovsky showed that several real TLS implementations were vulnerable, allowing extraction of server private keys.

The defence is a short, mandatory check, described as partial public-key validation in NIST Special Publication 800-56A: confirm the encoding is well formed, both coordinates are below the field prime, the point satisfies the P-256 equation and it is not the point at infinity. Because P-256's group has prime order, a point that passes these checks is automatically in the right subgroup. Every mature library performs this check; the danger lies in custom parsers and in code that skips it for speed.

A received P-256 point is used only after its encoding, coordinate range and curve equation all check out. Accepting a P-256 point Encoding valid?65 or 33 bytes, right flag Coordinates below p?x and y in range Rejectnever compute with it On the curve?and not infinity Acceptprime order: done Yes No Yes No No Yes
A received P-256 point is used only after its encoding, coordinate range and curve equation all check out.
⚠️
Invalid points leak keys. Skipping the on-curve check lets an attacker run your secret through a weaker curve of their choosing.

Input rules for X25519: clamping and the all-zero check

X25519 takes a different approach: instead of asking the implementation to validate input, it is designed so that every 32-byte string is safe to process. Because only x-coordinates are used, an input that is not on Curve25519 lies on a closely related curve called its quadratic twist, and Curve25519 was chosen so that its twist is also secure, a property called twist security. There is no need to check whether a point is on the curve.

The cofactor of 8 is handled by clamping. Before use, the private key's three lowest bits are cleared, which makes it a multiple of 8, its highest bit is cleared and the second-highest set. Multiplying any point of small order by a multiple of 8 lands on the identity, so a small-subgroup input cannot leak the private key's low bits the way the toy Diffie-Hellman example leaks parity. Fixing the top bit gives every key the same length, which keeps the ladder's running time uniform.

What an attacker can still do is send one of the few points of small order, which forces the shared result to all zeros. That does not reveal the private key, but it does mean the attacker knows the shared secret, which can matter in protocols where a party is supposed to contribute fresh randomness. RFC 7748 notes that implementations may check for an all-zero output, and TLS 1.3 requires that check for X25519 and an abort if it occurs.

Choosing between them

For a new system without external constraints, X25519 is usually the simpler and safer choice to implement correctly: fixed-size encodings, no point validation, a ladder that is naturally constant time and a design that tolerates hostile input. It is also the most common key share in TLS 1.3 handshakes, and it forms the classical half of the widely deployed hybrid post-quantum key share that pairs it with a lattice-based mechanism.

P-256 remains essential where compliance, hardware or ecosystem support requires it. Many hardware security modules, smart cards and certified cryptographic modules have long supported the NIST curves first, and regulated environments have historically mandated them; current certification rules should always be checked rather than assumed. A well-reviewed P-256 implementation that validates every point is perfectly sound.

The choice that matters most is not the curve but the implementation. Use a mature library, let it parse and validate keys, and never hand-write scalar multiplication.

Decisions for practitioners and AI agents

Responsibilities for safe ECDH are spread across layers, and failures happen where one layer assumes another has done the work. The cryptographic library owns constant-time arithmetic and input validation. The protocol owns the all-zero check, transcript binding and key derivation. The platform owns configuration, which curves are offered and where private keys live. Application code, including code written by assistants, owns not bypassing any of the above.

AI agents depend on all four layers on every call to a tool server, model endpoint or peer agent. Platform teams should standardise agent egress on TLS 1.3 with X25519, or the hybrid key share where available, and P-256 where compliance requires it; keep any long-term ECDH or signing keys in a key store or hardware security module; and monitor for libraries that are outdated or configured to reuse ephemeral keys. When agents generate integration code, watch for hand-rolled point parsing, keys loaded from environment variables and helper functions that 'decode a public key' without validating it. Those lines run perfectly in testing and remove the very checks that keep a private key private.

Safe ECDH needs each layer to do its own job; gaps appear when one layer assumes another has checked. Who checks what Library Protocol Platform Agent code Constant-time ladderand point validation All-zero checktranscript and HKDF Curves and key storeTLS 1.3, no key reuse No bypassesno hand-rolled parsing
Safe ECDH needs each layer to do its own job; gaps appear when one layer assumes another has checked.
  • Prefer X25519 for new systems; use P-256 where compliance or hardware requires it.
  • Validate every received P-256 point: encoding, range, curve equation.
  • Abort on an all-zero X25519 result in protocols that require it.
  • Use constant-time libraries; never hand-write scalar multiplication.
  • Keep long-term keys in a key store and never reuse ephemeral ones.

Key takeaways

  • P-256 and X25519 both give about 128-bit classical security, but differ in curve form, group structure, encoding and input rules.
  • Field primes are chosen for fast reduction, and point formulas use the curve's leading coefficient but not its constant, which matters for validation.
  • The Montgomery ladder performs the same add-and-double for every bit, making constant-time scalar multiplication natural, especially for X25519.
  • P-256 points must be validated, encoding, range and curve equation, or invalid-curve attacks can extract a reused private key.
  • X25519 accepts any 32 bytes thanks to twist security and clamping, but protocols such as TLS 1.3 must reject an all-zero shared result.
  • For AI agents, safety comes from TLS 1.3 configuration, mature libraries and key stores, never from hand-rolled curve code.

Glossary

ECDH
Elliptic-curve Diffie-Hellman, key agreement using scalar multiplication of points on an elliptic curve.
Scalar multiplication
Adding a point to itself a secret number of times, easy forward and hard to reverse.
Montgomery ladder
A method that performs one addition and one doubling for every bit, so its operations do not depend on the secret.
Constant time
Implementation style in which timing and memory access patterns do not depend on secret values.
Cofactor
The ratio between a curve's total number of points and the size of the large prime-order subgroup used.
Clamping
Fixing certain bits of an X25519 private key so it is a multiple of the cofactor and has a fixed length.
Invalid-curve attack
Sending points that lie on a different, weaker curve so a victim's computation leaks key bits.
Twist security
The property that inputs falling on a curve's quadratic twist are also safe to process.
Point validation
Checking that a received public key is well encoded, in range and on the expected curve before use.

Further reading

  1. RFC 7748: Elliptic Curves for Security (IETF)
  2. SP 800-186: Recommendations for Discrete Logarithm-based Cryptography: Elliptic Curve Domain Parameters (NIST)
  3. SP 800-56A Rev. 3: Recommendation for Pair-Wise Key-Establishment Schemes Using Discrete Logarithm Cryptography (NIST)
  4. RFC 8446: The Transport Layer Security (TLS) Protocol Version 1.3 (IETF)