Derive a key (and not store it) from a passphrase, to be used with AES

Question

This is more of a request of auditing my approach, thus asking here might not the best way, and I apologize for that, but I really don't know how to otherwise talk to cryptography experts. I have some basic knowledge about crypto and grasp (I hope!) the fundamental pieces, but I also feel that it's easy to overlook some aspect when putting all together. I'll strive to make this question the more general as possible, so to help others in similar situations.

Scenario

Imagine an application that is responsible handling AES-encrypted files. Upon startup it asks the user for a passphrase, and it derives a suitable key from it, which is kept in memory for the lifespan of the process and never stored persistently. The application uses this key to encrypt/decrypt files on some persistent storage.

The following assumptions hold:

the application runs on a trusted device, thus it is assumed that the derived key is not leaked nor read from memory;
the source code of the application is publicly available;
the persistent storage is untrusted (meaning that it can be leaked).

Encryption

AES-128 is used for encryption. A cryptographically secure random IV is generated for, and written to each file, followed by the actual ciphertext.

Key derivation

PBKDF2 is used to derive the AES key, with the following parameters:

an hardcoded salt, whose only purpose is to avoid the use of precomputed table to try to obtain the passphrase, the application is already compromised if the derived key is somehow leaked;
a hardcoded number of iterations set to a quite high value, say 1,000,000;
a key length of 16 (for AES-128);
SHA 512 as the hash function.

Is it better in this case something derived from the password as a salt? I know it's bad practice when storing hashes as it would nullify the purpose of the salt in the first place. But here it would prevent the hypothetical scenario in which a motivated attacker starts building a precomputed table with the hardcoded salt.

Is SHA 512 a good choice here?

Possible attacks (that I can think of)

Assuming an attacker to get a hold of some encrypted file, they can attempt to:

bruteforce the 128 bit key until some meaningful content comes out (unfeasible);
bruteforce the passphrase until some meaningful content comes out (unfeasible, due to the deliberate slowness of PBKDF2, unless a trivial passphrase is used).

Should I be aware of any other possible attacks?

Additional questions

Is this approach sound?

Can it be improved in some trivial way?

Welcome to the community. The real cryptography experts reside at Cryptography StackExchange, but this seems like more of a security question so it's fine — Sir Muffington, Jul 13 '23 at 17:09
Thank you @SirMuffington! I thought about it, but eventually decided for this one being this a more practical question. — cYrus, Jul 13 '23 at 19:38
WRT using a hardcoded salt - If many passwords are hashed using the same salt, then at some point, it may become feasible for an attacker to compute a rainbow table of passwords hashed using this salt. See https://security.stackexchange.com/questions/262314/does-recalculating-per-user-salt-every-time-a-user-changes-password-make-any-dif for more info. You may want to consider using a different randomly generated salt for each password, and storing this salt in the file along with the ciphertext, as you are doing for the iv. This is how openssl does it (see my answer below). — mti2935, Jul 14 '23 at 10:50
The actual problem of reusing salts is that an attacker can brute-force all derived keys with the same salt at once. For each passphrase candidate, they only have to do the expensive PKBKDF2 calculations once and can then verify the results against all keys/ciphertexts. In contrast, when the salts are random and unique, this forces an attacker to perform the expensive calculations for every key, because each key is parameterized with a different salt. — Ja1024, Jul 14 '23 at 12:23
As to rainbow/lookup tables, it's unclear how relevant these are today. Current GPUs, ASICs and FPGAs can calculate billions of SHA2 hashes per second, so it's hardly useful to build large tables in advance. Historically, the threat of precomputed rainbow tables may have been a valid argument for salts, but with current hardware, this is no longer true. — Ja1024, Jul 14 '23 at 12:31
@Ja1024 that's true for SHA2, but what about slow KDFs, like for example PBKDF2? — cYrus, Jul 14 '23 at 12:41
@cYrus: This is especially true for password-based KDFs, because an attacker can save an enormous amount of calculations if salts are reused. Besides that, PKBKDF2 is typically based on iterative applications of hash functions like SHA-2. — Ja1024, Jul 14 '23 at 16:49
With AES-128, make sure to use a suitable block cipher mode. Could be worth a separate question, but I'd suggest something that provides authentication and nonce-reuse-resistance, like AES-GCM-SIV. — jpa, Jul 14 '23 at 18:53

score 8 · Answer 1 · answered Jul 13 '23 at 18:01

Your approach is fine, but you are missing one thing: changing passwords. I would recommend changing PBKDF2 with a more modern KDF, like Argon2.

With your approach, you will need to decrypt and reencrypt every single file when the user wants to change its password. Other issue is when you want two or more people to have access to the same file.

You could implement Envelope Encryption, and will have 3 keys:

User AES Key: Generated by the KDF: recommended ones are PBKDF2, Argon2, Scrypt. Never stored anywhere.
Storage Master Key: Random AES key, encrypted by the User AES key. This is used to decrypt a key for each file. If the user changes its password, this key must be decrypted by the older password, and reencrypted with the new one. Always stored encrypted.
File Encryption Key: Unique random AES key for each file. Encrypted with the Storage Master Key, and stored encrypted along with the encrypted file.

When the user wants to decrypt its file, you run the KDF with its password, get the AES key, decrypt the Storage Key, use the decrypted Storage Key to decrypt the file key, and use the decrypted stored key to decrypt the file.

This way when the user wants to change its password, you only need to reencrypt the Storage Master Key. Without that intermediary key, you would either encrypt every file with the same key (and encrypt the file key with the KDF-generated key), or you would have to reencrypt every single file (when using only the KDF-generated key).

This allows you to implement multi-user encryption: encrypt each file with a file-key, and encrypt the file-key with the Storage Key of every user you want to grant access. To remove access from someone, just delete its encrypted file-key from the file header (or table, or whatever you use to store the keys).

You made a good point about the possibility of changing the password, it shouldn't be a big concern in my specific use case, but certainly something to consider. — cYrus, Jul 14 '23 at 09:16

score 4 · Answer 2 · answered Jul 13 '23 at 18:56

Your approach is very similar to the method that openssl enc uses to encrypt files with AES, using a key derived from a password. The salt used for key derivation is created randomly when the file is encrypted, then this salt is stored along with the ciphertext in the encrypted file. When the file is decrypted, the program derives the key using the password, and the salt from the encrypted file.

See https://crypto.stackexchange.com/questions/3298/is-there-a-standard-for-openssl-interoperable-aes-encryption/79855 for more info.

score 3 · Answer 3 · answered Jul 13 '23 at 18:49

3

Implementing file encryption properly is difficult, and there are countless possibilities for subtle mistakes which can compromise the security of the system. Even professionally written file encryption solutions like EncFS or ecryptfs have shown major problems in audits and had to go through many iterations to fix those issues (or have been completely abandoned).

To name just a few problems I see with your approach:

PBKDF2 is obsolete and fairly weak.
You misunderstand the purpose of salts. The main purpose is to prevent parallelized brute-force attacks: If every derived key has a different salt, this forces an attacker to brute-force each key individually, since the calculations cannot be reused across keys. However, if all keys are derived with a single salt as you propose, then an attacker has to try each potential password only once for all users.
Hard-coding all algorithms and parameters is a bad idea. You want cryptographic agility, i.e., it should be easy to change algorithms and parameters whenever needed.
In your attacks, you only consider a passive attacker who obtains a file and tries to obtain the plaintext through brute-force. What about active attackers who are able to manipulate the ciphertext or swap out files?

Since you said you only have basic cryptographic knowledge, I strongly suggest you use existing, mature software which has already been audited. The only valid reason to start from scratch is if you do this purely for learning and don't use the system to actually secure important files.

If this is a project for learning, start with the high-level concepts of existing solutions and read the audits to learn about common mistakes.

answered Jul 13 '23 at 18:49

Ja1024

5,769
14
21

Salts should definitely be generated randomly for each user/installation, unless there's some reason it's desired that entering the same password on two totally different installations of the app, in which case the salt should be derived from some user data (not necessarily the password; deriving it from e.g. an email address works). Also, good call out on ciphertext integrity; that's something that newbies often omit or get wrong, and the OP doesn't even list the block cipher mode they plan to use. – CBHacking Jul 14 '23 at 07:35
In the highlighted scenario, the probability of an attacker to obtain a derived key are very low, even more so the fact that they can obtain more than one. Using an hardcoded salt (vs not using a salt at all) would at least prevent an attacker to reuse a table precomputed with no salt, no? In any case, obtaining the passphrase doesn't give an attacker more advantage, they would still be able to decrypt the files. The only additional thing they can do it to target a specific user in case they reused the passphrase for something else. – cYrus Jul 14 '23 at 09:34
"What about active attackers who are able to manipulate the ciphertext or swap out files?" Eh, what about it? That was part of the question, if you care to elaborate... This is a good point to consider, I overlooked in my description because it's not my concern. With "untrusted" I meant that can be leaked, not modified. – cYrus Jul 14 '23 at 09:41
An attacker doesn't need the derived key to perform a brute-force attack against the passphrase or the ciphertext. That's the point. If the attacker has the ciphertexts and IVs, and the PBKDF2 parameters including the salt are hard-coded, they can immediately attack all passphrases and ciphertexts of all users at once: For each candidate passphrase p, they can calculate the corresponding derived key pbkdf2(prf, p, salt, it, len), where prf, salt, it, len are all hard-coded and the same for every user. Then they can go through all ciphertexts and try to decrypt them.

Ja1024

Jul 14 '23 at 10:07

Whenever they succeed, they've successfully obtained a key through brute-force. Note that the issue here is that all users can be attacked in parallel. This would be impossible if each user had a randomly generated individual salt, because then the attacker would have to do the expensive KDF calculations for each user. Precompuated tables are much less relevant, because hardware is so fast now that the hash calculations can be done ad-hoc.

– Ja1024 Jul 14 '23 at 10:11