Userfriendly

Author Topic: Main Problems  (Read 18600 times)

Offline AgAiNaWaY

  • Codemonkey
  • *****
  • Posts: 331
    • View Profile
Main Problems
« on: January 20, 2003, 10:20:33 am »
Basicly i see two main problems.

1. If a server connects two time to the same network they will collide or at last their nicks will collide.

I have no clue how to fix/avoid it. (without breaking something)

2. Where to send a message ? If you send it to both links the message will be send two times over the network. - We need a system which isnt to expensive to avoid this.
And what happens if a link stalls ? Lets say a server laggs for ~140 Seconds the Leaf sends his messages to him. Then he squits. - Are the messages lost ?
What happens if he sends the rescuing ping and everything gets send ?

Possible solution: Every message needs to carry an unique id. And the servers are counting them up If the server receives a message with an id he allready had seen ... he may only forward it .. or drop. But how many ids do we need ? And how much Traffic will this cost ? .. Well it zipped but its more.
And if a server get squited and relinked he needs to became the current id.
The ID may be evaluated with TS x somethingwhichrandomlychanges or something like this

Greets

Mathias

Offline ShadowMaster

  • Chief Codemonkey
  • Administrator
  • ********
  • Posts: 3135
    • View Profile
    • http://www.shadow-realm.org/
Main Problems
« Reply #1 on: January 20, 2003, 08:02:07 pm »
[list=1]
  • The linking protocol needs to be redesigned to allow links to tell the other link if they are a primary link or a redundant backup link. No synch information will be sent over the redundant link(s) so no nickname collides will occur that way.
  • Again see above, messages will only be sent over the primary link. A message cache and message id's will need to be created. The protocol would be best of being optimized using uniqueid's for nicks (see Hybrid 7 or Bahamut 1.6 for more info) aswell.
  • [/list:o]The real question becomes how do we solve the issue with how to acknowledge recieved messages. We need a message cache obviously, and we need a system to keep track of what messages have gone trough. This doesnt only apply to /msg and /notice but everything the IRCd sends. Else we will end up with desynchs fast.
    Each message will have to be marked with an unique id which allows the server to keep track of it. All servers needs to be marked with an unique id, and the message needs to be marked with an unique id to determine which server this message is headed for so that other servers know what to do with the message in the event of the primary link going down. Remember, we dont only have to think about the leaf server sending info, but we also need to think about servers at the far end of the network sending to a redundantly linked server which may go down while the message is enroute.

    Now other than that, we would be very wize to look at optimizing the server <-> server protocol further to reduce traffic as much as possible seeing as we are likely to increase bandwith usage compared to today. Not that it will be really noticeable as we are talking low data amounts. But tokenizing the protocol (see bahamut 1.6), introducing unique client id's and unique server id's will allow for greatly reduced traffic.

    It will make the protocol harder to "get info" for someone writing 3rd party apps, but i belive its worth it. Investigating both Hybrid7, IRCu and Bahamut 1.6 for ideas and solutions is a must.
Search before posting - No support by PM or IM

Offline AgAiNaWaY

  • Codemonkey
  • *****
  • Posts: 331
    • View Profile
Main Problems
« Reply #2 on: January 21, 2003, 08:45:52 pm »
Quote from: "ShadowMaster"
[list=1]
  • The linking protocol needs to be redesigned to allow links to tell the other link if they are a primary link or a redundant backup link. No synch information will be sent over the redundant link(s) so no nickname collides will occur that way.
  • Again see above, messages will only be sent over the primary link. A message cache and message id's will need to be created. The protocol would be best of being optimized using uniqueid's for nicks (see Hybrid 7 or Bahamut 1.6 for more info) aswell.
  • [/list:o]


If we give any server message an unique id and letting the server decide if he sends the message or not we get the real advantages of it.
And we will never have any loose or lagg/stalled messages at any time

Quote
The real question becomes how do we solve the issue with how to acknowledge recieved messages. We need a message cache obviously, and we need a system to keep track of what messages have gone trough. This doesnt only apply to /msg and /notice but everything the IRCd sends. Else we will end up with desynchs fast.


We need no ACK.

Every Server has a Counter i.e. 5454654 incomming messages having an id expected is 5454655 if he gets a message with id 5454654 he may drop it ... it has been already send from the other link side. Or he may forward it to the next hub... But i see no chances for desyncs at this point.
But incrased traffic for servers with more than one link but this is the price we pay.

Quote
Each message will have to be marked with an unique id which allows the server to keep track of it. All servers needs to be marked with an unique id, and the message needs to be marked with an unique id to determine which server this message is headed for so that other servers know what to do with the message in the event of the primary link going down. Remember, we dont only have to think about the leaf server sending info, but we also need to think about servers at the far end of the network sending to a redundantly linked server which may go down while the message is enroute.


Here comes an other problem i didnt noticed it befor due to normal network lagg .. what happens if a server send an message with an id which already exists ? ... It means every server need to have an own id ..  - it becomes ugly

Quote
Now other than that, we would be very wize to look at optimizing the server <-> server protocol further to reduce traffic as much as possible seeing as we are likely to increase bandwith usage compared to today. Not that it will be really noticeable as we are talking low data amounts. But tokenizing the protocol (see bahamut 1.6), introducing unique client id's and unique server id's will allow for greatly reduced traffic.


Maybe its necessary to get this work.

Quote
It will make the protocol harder to "get info" for someone writing 3rd party apps, but i belive its worth it. Investigating both Hybrid7, IRCu and Bahamut 1.6 for ideas and solutions is a must.


I agree with you :)

Offline ShadowMaster

  • Chief Codemonkey
  • Administrator
  • ********
  • Posts: 3135
    • View Profile
    • http://www.shadow-realm.org/
Main Problems
« Reply #3 on: January 22, 2003, 08:56:29 am »
Quote from: "AgAiNaWaY"


We need no ACK.

Every Server has a Counter i.e. 5454654 incomming messages having an id expected is 5454655 if he gets a message with id 5454654 he may drop it ... it has been already send from the other link side. Or he may forward it to the next hub... But i see no chances for desyncs at this point.
But incrased traffic for servers with more than one link but this is the price we pay.


On the contrary, we need an ACK cause we wont be wasting tons of bandwith by sending all data to all our links when only one of our links need that data, not to mention that we need to know if our message destined for "someserver" was accepted by our primary link or not. They may be several approaches to this, and i havent looked to closely at any of them as of yet. Jasper is going to be heading this work, but im throwing thoughts and ideas around :)

Quote

Here comes an other problem i didnt noticed it befor due to normal network lagg .. what happens if a server send an message with an id which already exists ? ... It means every server need to have an own id ..  - it becomes ugly


Actually it becomes much cleaner as instead of having to use myserver.area.region.somelongdomain.com we can now use a short numeric for prefixes.
Search before posting - No support by PM or IM

Offline AgAiNaWaY

  • Codemonkey
  • *****
  • Posts: 331
    • View Profile
Main Problems
« Reply #4 on: January 22, 2003, 06:03:13 pm »
Im very sure that we need to decide between lagg and traffic.

But this depends between every network ... how about building both .. a low traffic variant and a low-lagg variant. Or using both but not network wide. I´ll think a little bit about this...

I want to say is if we are waiting for an ACK every message has a lagg of 2 x latency. And this can be much for some links.

Offline ShadowMaster

  • Chief Codemonkey
  • Administrator
  • ********
  • Posts: 3135
    • View Profile
    • http://www.shadow-realm.org/
Main Problems
« Reply #5 on: January 22, 2003, 06:08:12 pm »
The lag wil only occur for messages which are lost enroute to the uplink server, ie the link is failing, in which case we are having slightly dela:badword: messages as they are resent to the new primary link instead of completely lost messages and a netsplit. Its simply not feasible to flood all our links with all messages destined for the same server.
Search before posting - No support by PM or IM

Offline AgAiNaWaY

  • Codemonkey
  • *****
  • Posts: 331
    • View Profile
Main Problems
« Reply #6 on: January 22, 2003, 07:26:49 pm »
Argh ! I misunderstood this.

You´re righht. But now we need to cache for about 10 seconds... how big will it be ?

Offline fish

  • Codemonkey
  • ***
  • Posts: 56
    • View Profile
Main Problems
« Reply #7 on: February 15, 2003, 05:16:12 pm »
Just my two cents here.
If your going to have reduntant links, why not utilize them (and not be, in this case, reduntant, but actually dual links)
Do it much the same way that BGP does on the internet. BGP maintains a list of the different paths available to each server, and always utilizes the shortest path to the destination.

in essence, each server would maintain a list of each server that is available on each link, and the number of hops each server on the network is from that link. eg:
Link1:
hub1 - 1 hop
Hub2 - 2 hops
Link2:
Hub1: 2 hops
Hub2 - 1 hop

In this arrangement, you could also introduce a "prefered Metric" so when two destinations have the same hops on each link, you can say link1 is prefered, as its lag is lower or whatever.


If you have leaf1 connected to hub1 and hub2, and with your current proposal, only the leaf1->hub1 link is active, and a user on leaf1 sends a message to a user on hub2, its not taking advantage of the leaf1->hub2

I think you have to think about this as both a reduntant change, and also improving the current server<->server routing.

as for the message ID's and ACK'ing, I believe you going to need to do that, but to reduce bandwidth why not set a threshold, say:
for every 5 messages recieved, send a ack
or
if you only have 3 messages so far, and its been longer than 10 seconds, then send a ack for those three messages.

and as Shadow Mentioned, Tokenising the messages structures is probably a good way to reduce bandwith as well... Considering that some messages can use upto (esitmation - 64 Max Length Servername + 8 chars for the command + another 64 (probably more) for the target info) 130 Bytes just for the prefix, command and target information, by using client and server numerics instead of full names, and tokenising the command portion of the message, could probably get it under 20 bytes (of course, ziplinks do help a lot, but ziplinks are a attempt to fix a symtom, and not the cause of the problem.)
Fish
-------------------------------------------------------------------
NeoStats Lead Developer (http://www.neostats.net/)
Ultimate3.x Developer (http://www.shadow-realm.org/)

Offline OrPhEuS

  • Here a lot
  • ***
  • Posts: 53
    • View Profile
Main Problems
« Reply #8 on: February 14, 2004, 02:11:12 pm »
I agree with Fish on this one, though would rather make it based on OSPF. Simply because this is a better example, then BGP which has more additions to let people decide what traffic you want to allow and which not.
And OSPF has also the implementation for preferred links. But also something important in my eyes is that two routes to a server which are both best, can be equally used using OSPF, so load balancing the traffic over the different hubs, instead of always everything over the primary one.

for the message ID's and ACK'ing, we will surely need this, but we will need to ACK on every message. Why? Simple to take care we ain't loosing them, and especially when they can be send over different routes to the same server.

What would happen in my eyes is this:

We got the following network:

Hub1
Hub2
Hub3

Leaf1
Leaf2
Leaf3

All hubs are connected to each other.
Leaf1 is connected to Hub1 and Hub2 and Hub3.
Leaf2 is connected to Hub1 and Hub2.
Leaf3 is connected to Hub1 and Hub3.

As soon as Leaf1 has to deliver a message to a client on Leaf3 this will happen:
Leaf1 checks his routing table, ok, either over Hub1 or Hub3. Hub1 is the fastest, let's send it over that one. Send message to Hub1.
Hub1 receives the message and sends an ACK to Leaf1, which removes the message from his cache.
Hub1 checks his routing table, straight to Leaf3 or over Hub3. In a normal situation it would take straight to Leaf3, though if that connection is lagging, it could send over Hub3, if that one announces a better roundtrip. Send message to Leaf3.
Leaf3 receives the message and sends an ACK to Hub1, which removes the message from his cache.
End of the message travelling around.

And yes this means that at some networks many ACK's will be send, but we can't risk it that we would loose messages.

Ofcourse it would be possible to say, well we drop messages we have had, because we recognize the ID. but how long will we have to hold those ID's in cache? That could start taking up enormous amounts of memory, especially on the bigger networks.
Founder / Network Admin @ After-All
www.After-All.org

Offline erich

  • Needs to get out more
  • ****
  • Posts: 132
    • View Profile
Main Problems
« Reply #9 on: July 01, 2005, 12:47:26 pm »
Damn... this section is dead  :oops:

Lets BUMP ones to restart the project. Maybe someone figured it all out by now  :idea:


/me hasn't
Network Founder @ After-All.org
http://www.After-All.org

Offline BarkerJr

  • Here a lot
  • ***
  • Posts: 97
    • View Profile
    • Hollimyer IRC Network
Main Problems
« Reply #10 on: July 01, 2005, 02:46:13 pm »
Quote from: "ShadowMaster"
Now other than that, we would be very wize to look at optimizing the server <-> server protocol further to reduce traffic as much as possible seeing as we are likely to increase bandwith usage compared to today. Not that it will be really noticeable as we are talking low data amounts. But tokenizing the protocol (see bahamut 1.6), introducing unique client id's and unique server id's will allow for greatly reduced traffic.

It will make the protocol harder to "get info" for someone writing 3rd party apps, but i belive its worth it. Investigating both Hybrid7, IRCu and Bahamut 1.6 for ideas and solutions is a must.


I believe that some IRCds have both, for backwards compatibility.  They can send tokenized to one server while sending expanded to the other.  This allows for services on localhost to the hub using the expanded protocol without impact to server throughput.

Offline erich

  • Needs to get out more
  • ****
  • Posts: 132
    • View Profile
Main Problems
« Reply #11 on: July 01, 2005, 11:42:39 pm »
Quote from: "BarkerJr"
I believe that some IRCds have both, for backwards compatibility.  They can send tokenized to one server while sending expanded to the other.  This allows for services on localhost to the hub using the expanded protocol without impact to server throughput.


Can you check which or maybe you know if you think hard  :wink:
Network Founder @ After-All.org
http://www.After-All.org

Offline BarkerJr

  • Here a lot
  • ***
  • Posts: 97
    • View Profile
    • Hollimyer IRC Network
Main Problems
« Reply #12 on: July 02, 2005, 04:48:38 am »
I think Unreal does it.  If you look in Anope's config, there's an optional 'UseToken' directive.  I think that means you can decide if you want to use tokens, or not.

Offline trystan

  • First timer
  • *
  • Posts: 9
    • View Profile
    • http://denora.nomadirc.net
Main Problems
« Reply #13 on: July 02, 2005, 11:12:36 am »
Haven't read the whole thread so won't comment on most of this... to Barker

What Unreal does is in the linking they send the PROTOCTL (CAPAB in Ultimate3), in this they send the "TOKEN" option, what this does is allows server to server to use a token ie.. "!" for PRIVMSG, by default all Unreal servers come this way, if the server linked in does not do tokens, the uplink feeds back the normal command.

What Anope does (seeing as I wrote the code for this).. with the option UseTokens enabled it will send the "TOKEN" in the capab line, and enable it to look for the token and the non-tokened command, then as its writting to the uplink it send the correct token or command.

This is not a server ID or user id.. these are more common in P10 (ircu) and TS6 ircds (hybrid, ratbox), these convert the user name or the server name to a 5 to 6 character base64 encoded string, they send send this string when the protocol calls for the user or server name to appear.. ie.. a privmsg comes out more like

ACAAA P ACAAB :HELLO!

P10 has gone as far as to replace the commands with tokens only

Now Unreal has started some work on this with their NS (Numeric Server), this replaces the server name with a Numeric base64 encoded number. However this seems to be from hub to leaf only, not from leaf to hub at the current time.

Yes many of the P10 or TS6 ircds support to some extent a leaf server that communicates without the SUID or UID and non token, but as they pass the information to the leafs its normally converted out to the protocol spec.. or at times they just drop the leaf

Offline erich

  • Needs to get out more
  • ****
  • Posts: 132
    • View Profile
Main Problems
« Reply #14 on: July 07, 2005, 05:26:09 pm »
Obviously it would be nice if UltimateIRCd would be using TS6, but I guess we have to try to make something ontop of the existing protocol in a routing level (if possible), so the routing isn't part of the protocol itself. TS6 really should be done by ShadowMaster and his brilliant team anyway, as its the core of ircd for sure.

Maybe it isn't possible to seperate the 2 parts of the protocol. But it would be nice if we could, as it would allow users to select whether they want to have it or not, even per link. This would be handy for BOPM, OPSB, Anope etc.
Network Founder @ After-All.org
http://www.After-All.org

 

anything
SimplePortal 2.3.7 © 2008-2022, SimplePortal