NetworkDesign: Difference between revisions

Jump to navigation Jump to search
← Older edit
Content deleted Content added
imported>Quisar
imported>Javydreamercsw
m Updated links to point to Git repository instead.
 
(162 intermediate revisions by 8 users not shown)
Line 1: Line 1:
{{Navigation for Marauroa Top|Internals}}
= Basic idea behind Networking =
{{Navigation for Marauroa Developers}}
The idea behind arianne's network protocol is to use a single stream of UDP packets between the server and the clients. Different kinds of in-game actions create different types of messages that are then interpreted at the opposite side in to meaningful data.


== UDP Packet Format ==
The network system is based on Messages being transmitted using UDP Packets. There are two types of data stream; one from the Server to the Client and another one from the Client to the Server.


Please note: This page explains the low level network communication. You won't need to bother with these implementation details if you want to use Marauroa to write a game. We document the network design anyway for contributors to Marauroa itself. And it is helpful for people porting Marauroa to other programming languages.
===UDP Client to Server communication stream===
Each message is contained in a SINGLE UDP Packet. There is no mechanism for recovery of lost or repeated messages.


==Messages==
<b>Each UDP Packet is composed of:</b>
Marauroa uses messages to communicate between client and server. The messages sent from the client to the server are prefixed with C2S and the ones sent from the server to the client use the prefix S2C.
* <i>Protocol version ( 1 byte )
* Type of Message ( 1 byte )
* Client ID ( 4 bytes )
* Rest of Message ( up to 1494 bytes )</i>


Each message is implemented in its own Java class in the package marauroa.common.net.message. You can lookup up the details about every message in the [http://stendhalgame.org/jenkins/job/marauroa_HEAD/javadoc/marauroa/common/net/message/package-summary.html javadoc]. If you want to port Marauroa to another programming language, you will need know how the message are serialized exactly. The easiest way to learn that is to look at the [http://arianne.cvs.sf.net/viewvc/arianne/marauroa/src/marauroa/common/net/message source code] of the readObject() and writeObject() methods.
===UDP Server to Client communication stream===
Each message is held in one or more UDP Packets and the UDP Packet format is as follows: <br>


There are for different client states of the game: connected, logged in, in game, logged out. Depending on the state different messages are valid:
<b>1st UDP Packet is composed of:</b>
* <i>Total number of packets (1 byte)
* Position of this message (1 byte)
* Signature of the message (1 byte)
* Protocol version (1 byte)
* Type of Message (1 byte)
* Client ID (4 bytes)
* Rest of Message (up to 1491 bytes)</i>


=== State connected ===
<b>All other UDP Packets making up the one message are composed of:</b>
* <i>Total number of packets (1 byte)
* Position of this message (1 byte)
* Signature of the message (1 byte)
* Rest of Message (up to 1497 bytes)</i>


[[Image:messages-connected.png|thumb|Messages used to securely login.]]
The idea is to send the serialized message inside several UDP Packets and then rebuild the full message on the Client. There is no kind of error recovery or retransmission.


The login process is a bit complicated because of security requirements. Don't be scared, just follow it step by step.
Messages are sent from the Server by simply serializing them. So the message itself contains the protocol version, the type of message and the Client id.


Onces the TCP connection is established the clients requests the RSA public key from the server using [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageC2SLoginRequestKey.java C2SLoginRequestKey]. The server checks the protocol version implicitly included in every message. If it is compatible, it replies with a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CLoginSendKey.java S2CLoginSendKey] including the RSA public key. It is composed of two bytes arrays: The first one contains the value of 'n', and the second the value of 'e'.
Receiving the message is a bit more complex. First we need to determine that we are running a compatible version of the protocol by comparing the protocol version with the expected version in the message. Once we have agreed that the protocol is compatible we read the type of message and we ask the Message Factory ( see [[RolePlayingDesign]]) to build an object of that type with the corresponding data. Once the message is built we simply store it in a queue of incoming messages waiting to be processed.


The client now computes a nonce (a random number) and sends its hash as byte array to the server in a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageC2SLoginSendPromise.java C2SLoginPromise] message. The server remembers the client nonce and answers with its own nonce in a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CLoginSendNonce.java S2CLoginSendNonce].
=Network Manager=
The Network Manager is our router that sends and receives messages to and from the network. The manager exposes the interfaces that allow:
* Reading a message from the network
* Sending a message to the network
* Finalizing the manager


Almost there: The client has now all the information it needs to actually send the [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageC2SLoginSendNonceNameAndPassword.java C2SLoginSendNonceNameAndPassword]: Its nonce, the username and the value rsaCrypt(xor(xor(client nonce, server nonce), password)). The first field is a bytes array containing the client nonce, the second one a string containing the username and the third one a byte array containing the encrypted password. On reception, the server checks that the hash he received at first is the hash of the nonce he just received. It then decodes the password field, and having the value of the client nonce and its nonce, it gets the value of the password.
The read operation is a blocking type operation so we have two options, either Polling (i.e. continually checking if data is there) or Blocking (i.e. only processing when data is actually available, otherwise sleeping).


The [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CLoginNACK.java S2CLoginNACK] message is sent from the server to the client to tell the client that its login request was rejected because the username or password is wrong, the account was banned or the server is full. The included result object will tell which of the cases prevented the login.
(<i>Polling is "the sequential interrogation of devices for various purposes, such as avoiding contention, determining operational status, or determining readiness to send or receive data." - Wikipedia ( http://en.wikipedia.org/wiki/Polling )</i>)


If the username/password combination, however, is correct then the Server must send a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CLoginACK.java S2CLoginACK] message to tell the client that the message has been correctly processed. It contains information about the last login, so that the user is able to recognize unauthorized usage of his account. The client state is changed to "logged in" in this case.
We choose Blocking because we don't want to waste CPU time Polling the network for messages, we just want to sleep until messages are available. Hence we create a Thread to read from the Network, let's call it Network Manager Read.
{{br}}


=== State logged in ===
Writing messages to the network can be simply coded as a method of Network Manager, as write is an operation that is non blocking by nature.


[[Image:messages-loggedin.png|thumb|Selecting a character and transmitting world meta data.]]
The Network Manager opens a Socket from which it will receive all the messages from the network. It will also write all the outbound messages to the network from this same socket. Note: Both write and read use the same Socket. (


The logging in stuff was complicated. But luckily things are getting much easier now. After the login completed successfully the server sends a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CServerInfo.java S2CServerInfo] message. It tells the client about what kind of server is running, and details on how to contact the server administrator (e.g. their email address). The message is composed of a List of strings of the form "attribute=value". In addition this message also contains the list of defined RPClasses.
To encapsulate all this we create both the Read and Write methods as inner classes of Network Manager.


Directly afterwards a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CCharacterList.java S2CCharacterList] message is sent from the server to the client. It offers a choice of character to play with. This feature models the behavior of having several characters associated with a single account. Each character name must be unique at server level, and it is assigned when the character is created.
<pre>
NetworkManager
{
socket
messages
pendingToSendMessages


Now the client picks one of the offered characters. Games that do not support multiple characters can pick the one that matches the account name. The choice is transmitted to the server using a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageC2SChooseCharacter.java C2SChooseCharacter] message. The name of the character must be one of the names listed in the character list.
NetworkManagerRead isa Thread
{
read socket
build message
store in messages
}


The server will check the selected character and reply for a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CChooseCharacterNACK.java S2CChooseCharacterNACK] if the choice was invalid. This implies that the client should send another C2SChooseCharacter message.
NetworkManagerWrite isa Thread
{
get from pendingToSendMessages
build UDP from message
send socket
}
}
</pre>


If the selection, however, was successful, a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CChooseCharacterACK.java ChooseCharacterACK] message is sent and the client state changed to "in game".
As you can see, messages are stored in a list when they are received. Hence access to the list must be synchronized.
{{br}}


=== State in game ===
Now lets get back to the interface as exposed to other objects. The Write method is immediate, just call it with the message to send, making sure that you have correctly filled SourceAddress and ClientID. The message will then be sent to the Client.


[[Image:messages-game.png|thumb|Messages sent while the game is active.]]
It is important to notice that as the transport is based on UDP there is no guarantee that the message is correctly sent. This situation is part of the protocol itself.


==== Regular Messages ====
The Read method is blocking, when you call the Read method it either returns a message from the queue or if the queue is empty the thread blocks (sleeps) until one arrives.


The [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CPerception.java S2CPerception] message is a message sent from the server to the client to notify the client about changes to the objects that exist near it. The message is based on the idea explained in the [[RolePlayingDesign#Perceptions|Delta Perception]] document.
That is the basic idea of the Network Manager; however, the manager gets a bit more complex as a result of the need to support Server to Client messages that are bigger than one UDP package. Note that the manager just sends the stream of packets once and doesn't confirm if any of the messages are received. Please refer to [[NetworkDesign#UDP_Packet_Format | UDP Packet Format]] for more info about it.


The message is composed of:
=Message Types=
* A type that can be DELTA or TOTAL
To communicate, the Client and Server of Marauroa use a stream of UDP packets. The message system belongs to the marauroa.common.net package, so refer to the code for a detailed knowledge of the system.
* A string indicating the name of the zone the perception is related to.
* A time stamp value that will be just one unit bigger than the previous perception
* A List of RPObject that contains added object
* A List of RPObject that contains modified added object attributes
* A List of RPObject that contains modified deleted object attributes
* A List of RPObject that contains deleted object
* A RPObject that describes the things the rest of players don't see about OUR own object.


Read the Delta perception algorithm to understand what it is for.
We have Client to Server, aka C2S, and Server to Client, aka S2C, messages. <br>
Most messages have two versions of themselves: the C2S and S2C versions.


Communication Model
<pre>
Client Server
------ ------
--Anytime:
Send C2S Login Request Key Send S2C Send Key
--onConnect:
Send C2S Login Promise Send S2C Send Nonce
Send C2S Login Send Nonce
Name and Password If C2S Login is correct
Send S2C Login ACK
else
Send S2C Login NACK
end if


The client sends [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageC2SKeepAlive.java C2SKeepAlive] messages regularly. If there has been no keep alive message for some time, the server will timeout the client.
Send S2C Character List


Send S2C Server Info


==== Static Content Transfer ====
Send Choose Character If character is correct
Send Choose Character ACK
else
Send Choose Character NACK
Finish


Perceptions are about dynamic content. But most games have some static data, too. Like maps, tilesets, sounds. The RPManager keeps track of situation in which the client might need some of this static data. A common case is the movement of the client from one zone to another.
--onTimer:
Send Perception ACK Send Perception


If the RPManager detects such a situation, it will offer the new content to the client using a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CTransferREQ.java S2CTransferREQ] message. The message is composed of an array of [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/TransferContent.java TransferContent] objects containing the name of each resource, its timestamp (or checksum) and if the resource is cacheable or not.
--on RPManager request:
Send TransferREQ
Send TranferACK
Send Transfer


The clients replies with a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageC2STransferACK.java C2STransferACK] acknowledging the offer. The message is composed of an array of TransferContent objects containing all the name of each resource and a flag indicating ack or not. Note: The C2STransferACK message is always sent, even if all flags indicate that no transfer should take place.
--onEvent:
Send Action Send Action ACK


If the clients has acknowledged the need for content to be transfer, the server sends a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CTransfer.java S2CTransfer]. The message contains a again an an array of TransferContent objects. This time, however, the actual data is included as well.
--onExit:
Send Logout If Client can logout
Send Logout ACK
else
Send Logout NAC
</pre>


==== Actions ====
Let's discuss in detail each message type.


Actions are commands sent from the client to the server using a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageC2SAction.java C2SAction] message. Example for commands are "move to the right", "look at that object", "attack that rat". It is up to the server to decide whether to execute the action or reject it.
==Message C2S Login Request Key==
{{br}}
This Login Message is sent from the Client to the Server to request the RSA public key of the server.


==Message S2C Login Send Key==
=== Logging Out ===
This Login Message os sent from the Server to the Client in response to a key request. It is composed of two bytes arrays. The first one contains the value of 'n', and the second the value of 'e'.


[[Image:messages-logout.png|thumb|The client sends a logout request and the server accepts or denies it.]]
==Message C2S Login Promise==
This is the first real action of authentification from the client. It computes a nonce (a random number) and sent a hash of its nonce to the server. This message contains the hash as a bytes array.


If the player is in some kind of combat it is often desirable to prevent him from saving his live by logging out. Therefore the client sends a logout request to the server and the game server can decide whether to accept or reject it. Of course the user can close the client window or force a disconnect of his Internet connection. But in these cases he will simple stay in a game unattended until the timeout anyway.
==Message S2C Login Send Nonce==
The server responds to the client promise by computing a nonce and sending it in this message. This message contains the nonce as a bytes array.


The clients indicates that it wants to finish the session by sending a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageC2SLogout.java C2SLogout].
==Message C2S Login Send Nonce Name and Password==
The client now sends its nonce, its username and the value Crypt(xor(xor(client nonce, server nonce), password). This message contains 3 fields. The first one is a bytes array containing the client nonce, the second one a string containing the username and the third one a bytes array containing the encrypte password. On reception, the server checks that the hash he received at first is the hash of the nonce he just received. It then decode the password field, and having the value of the client nonce and its nonce, it gets the value of password. If the username/password combination is correct then the Server must send a Login ACK Message to indicate to the Client that the message has been correctly processed. However, if the username/password is wrong the Server will send a Login NACK (Not ACKnoledge).


The server can reply with a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CLogoutNACK.java S2CLogoutNACK] to reject the logout request. Or it confirms the request with a [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/MessageS2CLogoutACK.java S2CLogoutACK]. In this case the client state is changed to logged out.
==Message S2C Login ACK==
{{br}}
The LoginACK Message is sent from the Server to the Client to indicate that the login process has been completed correctly and that it can proceed to the next step. It has no params.


== Transmitting Messages over TCP ==
==Message S2C Login NACK==
The idea behind Arianne's network protocol is to use a single TCP stream between the server and the clients. Different kinds of in-game actions create different types of messages that are then interpreted at the opposite side in to meaningful data. TCP takes care of sorting the packets and retransmitting lost ones.
The LoginNACK Message is sent from the Server to the Client to tell the Client that its login message is incorrect because either the username or password is wrong or because Server has decided to reject the login request, e.g. if the Server is full or because the IP is banned.


Each message has a general header:
The message is composed of: Message indicating the resolution.
* Size of message (4 bytes)


* Protocol version (1 byte)
==Message S2C Character List==
* Type of message (1 byte)
The CharacterList Message is sent from the Server to the Client to give the Client a choice of character to play with. This feature models the behavior of having several characters associated with a single account ( see [[DesignOverview#What_is_Marauroa.3F | the Marauroa Feature List]] ).
* Client ID (4 bytes)
* Timestamp (4 bytes)


This header is followed by message specific data. Have a look at the [https://sourceforge.net/p/arianne/marauroa/ci/master/tree/src/marauroa/common/net/message/ source code] of the methods readObject() and writeObject() of the message in question.
The message is composed of: A List of Character names


==Network Manager==
Each character name must be unique at Server level, and it is assigned when the character is created.
The Network Manager is our router that sends and receives messages to and from the network. The manager exposes the interfaces that allow:
* Reading a message from the network
* Sending a message to the network
* Finalizing the manager


The read operation is a blocking type operation so we have two options, either polling (i.e. continually checking if data is there) or blocking (i.e. only processing when data is actually available, otherwise sleeping).
==Message S2C Server Info==
The Server Info Message is sent from the Server to the Client to tell the Client about what Server is running, and details on how to inform the Server administrator of any problems (e.g. their email address).


We choose blocking because we don't want to waste CPU time polling the network for messages, we just want to sleep until messages are available. Hence we create a Thread to read from the Network, let's call it NetworkManagerRead.
The message is composed of: A List of strings of the type "attribute=value"


Writing messages to the network can be simply coded as a method of Network Manager, as write is an operation that is non blocking by nature.
==Message C2S Choose Character==
The ChooseCharacter Message is sent from the Client to the Server to indicate what character the user wishes to play the game with.


The NetworkManager opens a Socket from which it will receive all the messages from the network. It will also write all the outbound messages to the network from this same socket. Note: Both write and read use the same Socket.
The message is composed of: the name of the character


To encapsulate all this we create both the Read and Write methods as inner classes of Network Manager.
The name of the character must be one of the names listed in the Character List Message or the Server will reply with a Choose Character NACK message. Once the character has been chosen and the acknoledge has been recieved the Client is ready to start the game.


<pre>
The Server will reply to this message with a Choose Character ACK or a Choose Character NACK message. The Client must have already logged in before receiving this message, if not the Server will send a Choose Character NACK.
NetworkManager
{
socket
messages
pendingToSendMessages


NetworkManagerRead isa Thread
Once a character is chosen, the only way to choose a new one is by logging out and logging in again.
{
read socket
build message
store in messages
}


NetworkManagerWrite isa Thread
==Message S2C Choose Character ACK==
{
The Choose Character ACK Message is sent from the Server to the Client to notify the Client that the character has been chosen.
get from pendingToSendMessages
serialize message
send socket
}
}
</pre>


As you can see, messages are stored in a list when they are received. Hence access to the list must be synchronized.
The message is composed of: the object.ID of the character

We need this value to track our own character.

==Message S2C Choose Character NACK==
The Choose Character NACK Message is sent from the Server to the Client to notify the Client that the character has not been chosen. This also implies to the Client that it should send another ChooseCharacter Message.

It has no params.

==Message C2S Logout==
The Logout Message is sent from the Client to the Server to indicate that it wants to finish the session. It has no params.

If the Client can logout the Server will send a Logout ACK Message and the Client can safely close the connection, but if the Server sends a Logout NACK Message, the Client should keep playing as disconnecting at that moment will leave its avatar in the game uncontrolled ( idle ).

==Message S2C Logout ACK==
The Logout ACK Message is sent from the Server to the Client to notify the Client that the logout has been accepted.

It has no params.

==Message S2C Logout NACK==
The Logout NACK Message is sent from the Server to the Client to tell the Client that the logout has not been accepted, possibly because something related to the RP entity, for example, combat is in progress.

It has no params.

==Message C2S Action==
The Action message is sent from the Client to the Server to notify the Server of the Clients desire to execute an action. The message is simply composed of a single action.

==Message S2C Action ACK==
The Action ACK is sent from the Server to the Client to identify that the action has been received in the Server. It doesn't acknowledge that the action has been accepted by the RP Manager, it just means that the action successfully arrived at the Server.

It has 1 param: action_id - is used to identify to which action you referred. In Marauroa you can command several actions per turn.

==Message S2C Perception==
The Perception message is a message sent from the Server to the Client to notify the Client about changes to the objects that exist near it. The message is based on the idea explained in the [[RolePlayingDesign#Perceptions| Delta Perception document]].

The message is composed of:
* A type that can be DELTA or TOTAL
* A string indicating the name of the zone the perception is related to.
* A time stamp value that will be just one unit bigger than the previous perception
* A List of RPObject that contains added object
* A List of RPObject that contains modified added object attributes
* A List of RPObject that contains modified deleted object attributes
* A List of RPObject that contains deleted object
* A RPObject that describes the things the rest of players don't see about OUR own object.

Read the Delta perception algorithm to understand what it is for.

==Message C2S Perception ACK==
The perception ACK message is sent from the Client to the Server. We use this message to inform the Server that the Client is still there and listening. It has no params.

==Message S2C TransferREQ==
The TransferREQ message is a message sent from the Server to the Client to notify the Client of the server's desire to send content useful for game play.

The message is composed of: An array of TransferContent objects containing the name of each resource, its timestamp and if the resource is cacheable or not.

==Message C2S TransferACK==
The TransferACK message is a message sent from the Client to the Server to notify the Server about what of the given TransferContent elements recieved from TransferREQ should be sent.

The message is composed of: An array of TransferContent objects containing all the name of each resource and a flag indicating ack or not.

==Message S2C Transfer==
The Transfer message is a message sent from the Server to the Client to send the Client content useful for the game where play will happen.

The message is composed of: An array of TransferContent objects containing all the name of each resource, its timestamp, a flag indicating if the resource is cacheable or not and a byte array with the content itself.

= Versioning and port numbering =
A problem arises when we need to have two versions of Marauroa on the same machine. This happens for example when you run a release version and a development version on the same machine.


Now lets get back to the interface as exposed to other objects. The write method is immediate, just call it with the message to send, making sure that you have correctly filled SourceAddress and ClientID. The message will then be sent to the Client.
So to avoid port collisions we will use different ports for each release. The release version will run on the standard game port bu the development version will run on the (standard game port)*10


The read method is blocking, when you call the read method it either returns a message from the queue or if the queue is empty the thread blocks (sleeps) until one arrives.
For example:
release version of gladiators runs at 3214
development version runs at 32140


That is the basic idea of the Network Manager. Note that the manager just sends the stream of packets once and doesn't confirm if any of the messages are received. TCP takes care of that.
To allow different servers of Marauroa to run on the same machine running different games we will assign an unique port number to each game.


gladiators runs at 3214
mapacman runs at 3215
stendhal runs at 3216


[[Category:Marauroa]]
Feel free to follow or disgregard these port rules if you wish. They may not apply to you.
{{#breadcrumbs: [[Marauroa]] | [[Navigation for Marauroa Developers|Internals]] | [[NetworkDesign|Network Design]] }}
Retrieved from "https://meet.konsthol.eu/w/index.php?title=NetworkDesign"