I think your basically on the right track. Your client needs to provide some
token which your server will recognise or accept as proof that the client is
legitimate. How to do this depends on a number of factors and what the risks are
you are trying to protect against. There is no one single solution.
Some of the things to consider may include
Complexity. This is possibly the biggest threat to both security and
reliability. The more complex the solution is, the more difficult it is to
prove correct. This can mean both an increase in security flaws as well as
just normal bugs and will result in higher maintenance costs. As the saying
goes "everything should be as simple as possible, but no simpler".
Level of assurance. What is it you need to be assured about. Do you just need
to know the client will behave correctly i.e. make requests which the server
can understand and respond to rather than just consume resources and possibly
degrade service or do you actually need to know it is a specific approved
client or perhaps an approved user or maybe even coming from an approved
source (IP address). If you need high levels of assurance, what do you need to
do to prevent clients from lying - trying to trick your server by providing
fake credentials and what do you need to do to prevent theft of credentials
(from sniffing network packets, MitM (man in the middle) attacks etc.
The robustness of your server. Your server should be liberal in what it will
accept and conservative in what it sends. This basically means your server
should handle input in such a way that it will recover from bad input
(incorrect format, corrupted data, etc) and not simple crash or exhibit
unexpected behaviour (such as dumping out sensitive data tot he client) when
provided with unexpected input and will be conservative in the responses it
sends (i.e. predictable and consistent).
The inherent value. What is the inherent value to you, your client and
others. This will help determine what level of protection you need to
consider. The likelihood of attack depends to some degree on the value (or
perceived value) of the service or data to an attacker. This can be difficult
to assess as you cannot always identify the motives an attacker may have. In
some cases, it is easy, such as when involving assets with a monetary
value. Other times, less so, such as when it involves 'bragging rights',
revenge or personal grudges or possibly some misguided belief or
understanding.
Understand the architecture and underlying technology. It is important to have
a good basic understanding of TCP/IP in order to identify the most appropriate
controls. For example, the difference between basic protocols (udp v tcp), the
basics of how the connection is made as this will determine which controls are
most appropriate - for example, understanding at what point during the
connection you can make a decision or even whether a certain decision is even
meaningful and what level of trust you can put in the information your being
given. For example, it is easy to spoof the source address of a UDP packet
because there is no two way connection handshake, but harder to do so with a
TCP socket because there is.
Avoid 'rolling your own'. Don't try to invent your own security solution. Use
an established and tried technique. In this respect, your question indicates
your heading in the right direction.
Consider some basic use cases such as a standard web site (essentially a server
with clients who connect to a specific port) compared to a pay-per-use service,
such as Audible. In the first case, most of the time, the server is less
interested in the specific individual or client. What the server wants to know
is that the client understands the protocol and sends commands the server
understands. In the second case, the server needs to know the client is
legitimate, the user is a know paid up subscriber and possibly that the client
is an approved client (in the case of Audible, the approved clients are able to
verify encryption keys so that the client can decrypt the audible book which has
DRM protection). In these two cases, the requirements are vary different and
will be addressed in different ways. In the first case, you really just need to
know the client understands your protocol and will likely behave in an
acceptable manner. in the second case, you need to protect against forgeries and
will likely need more complex functionality, such as encrypted communications,
shared encryption keys, login credentials etc.
In your case, where your experimenting and learning, you want to start of
simple. Defining a simple protocol which requires the client to send a known
'fact', such as a string or key is probably fine. However, it also depends on
the environment your working in. If your just experimenting on systems within a
LAN which has reasonable firewalls between you and the Internet, then you
probably only need to worry about systems on your LAN. You should ensure your
using non-privileged ports (i.e. above 1024) and avoid using ports which already
have a well defined or common use and don't leave your server operational when
not actively using it. If on the other hand your operating in a more accessible
environment, such as using a cloud based platform, then you may need to be a
little more defensive - log what is connecting, perhaps limit connections to the
IP addresses of your test clients etc. Note that advice not to roll your own
security solution can be ignored when you are just experimenting or learning. In
some cases, trying to first solve the problem yourself is not a bad way of
learning and can help in later understanding of why established solutions are
designed they way they are - just don't try to do it for a real application.
There are many books which describe how to secure server applications. Key
topics to cover would be things like network and application firewalls and
TCP/IP. common encryption and hashing techniques and the applicaitons/protocols
which use them (HTTPS, SSL/TLS, SSH, PGP/GPG). Have a look at existing
protocols. The HTTP/HTTPS protocol is a good starting point as it is relatively
simple. Don't get overwhelmed by the complexities, especially with respect to
things like encryption and hashing. These are vary complex topics, but you
really just need to understand the principals and how to apply them rather than
the vary technical detail. this is the main reason you should use a known and
tested approach rather than try to invent your own.
You may also find some of the following useful
- https://www.owasp.org
- https://www.feistyduck.com/books/bulletproof-ssl-and-tls/bulletproof-ssl-and-tls-introduction.pdf
- https://letsencrypt.org/
The OWASP site is particularly useful as it has lots of great information on
common security flaws/mistakes developers make and how to both detect and
prevent them. Much of this information can be generalised to any server/client
environment.
