so um thank you very much everyone for to for coming to this talk I'll just introduce the talk very briefly uh but I'm really grateful to Andrew for his keynote uh he set the scene really well because what I'm going to talk about in the next half hour or so is uh it's very practical focused but we're going to look at identifying threat actor infrastructure but we're going to look particularly at the Russian threat and we're going to focus very Nar on how they set up and run their disinformation operations so uh I'll introduce myself very briefly so as you just saw I'm a certified Sans instructor I teach SE 497 which is the the introductory oin class I also teach SE 587 the the advanced open source intelligence class uh away from ass Sans I work for an organization called protection Group International PGI uh we are a digital threats intelligence company and I spent a lot of time in the past couple of years uh looking at how nation states use the internet in a malicious way to attack our our societies and our our states so um let me go back one this is what I'm going to talk about today so it's always a little bit tricky with these conferences because some of you this might be the first time you've ever been to an oin conference and you may be very early in your career others of you have been doing this for a very very long time uh and are very technically skilled so there'll be something for everyone in this talk we're going to look at practical examples so uh one of the reasons I chose this topic is because when I teach classes or I give talks to different places one of the topics that always comes up on a student wish list is they usually want to get better at infrastructure investigations so by that I mean things like investigating websites domains attributing them and so on and uh security certificates SSL certificates is a really good place to start doing that so we're going to look at three case studies and as we go along uh we'll in increase the complexity of what we can do but security certificates are a hugely underutilized resource for oent investigators so hopefully over the next half hour uh we will begin to understand a little bit more about how we can use them of course if you have any questions uh just fire them into the slack and I'll pick them up at the end of The Talk so from the very beginning what is an SSL certificate why do I care uh what does it matter so um security certificates uh verify the identity of a website so when I connect to sans.org my browser needs to know that I really am connecting to sans.org and not some some spoof or some kind of imitation website uh and security certificates also allow me to make an encrypted connection to that website so my data is safe now every single security certificate or SSL certificate they're unique and every single one of them is made public uh in a public Ledger on a public Ledger we call them security certificate transparency logs so every single website that has ever existed even if it's been used for something malicious oh well maybe not that's ever existed but certainly in modern era um it doesn't matter if the website is malicious it doesn't matter if it's being used for fraud or for fishing it needs to have a security certificate and if it has a security certificate it has to be made public and that means that we as ENT practitioners have a huge uh resource of millions and millions and millions of security certificates uh that we can search through to map infrastructure to identify malicious behavior it's a very very underused resource so hopefully uh we'll shed some light on it in the next half hour other great things about security certificates as a data resource is not only are they all made public but once you have issued a security certificate doesn't matter how malicious you are as a threat actor it's out there it's public you can't alter uh what's been put out there so you can revoke a security certificate or you can issue a new one but once you put it out there as you'll see in our first case study you can't wind it back in once it's been published you can't see that so what we have available to us is this huge repository of public information about websites infrastructure uh that is freely available and it's also a very high quality uh resource so we're going to look at how to exploit this and we're going to focus specifically on Russian disinformation operations that have targeted the US and the UK and Europe in the last year or so so um so here's our first point why security certificates are useful they're really useful for attribution why are they useful for attribution well if I own a domain so if I own sans.org I have to prove that I own that domain before I can obtain a security certificate for it so I can't if I wanted to imitate gmail.com for example I can't just go I Steven Harris can't just go and get a security certificate for gmail.com I have to prove usually through um some kind of cryptographic test um that I actually own that domain so if someone obtains a security certificate for a domain they have to have proved that they own it so very strong uh attribution this is a very strong connection not anyone can just obtain a security certificate but supposing that I have multiple uh multiple domains that they own because I'm running a campaign that involves many many domains many different websites um one option I have is to obtain what we call a subject alternative name or a sand certificate nothing to do with Sans uh just coincidence there a subject alternative name so if I own lots of domains I can just roll them all into one certificate I don't have to do that but it does make life a little bit easier perhaps so what that means for us as practitioners is when we're looking at security certificates and I'll show you some examples of this in just a moment but when we're looking at these security certificates if we see lots of domains that all share the same certificate it's there is a very very high likelihood that those domains are owned and controlled by the same person because they have they have to have proved ownership of all of those domains very important as we'll see in our first case study so first example is a website which came online late in 2022 is since offline um it was called DC weekly.org and it it claimed to be a newspaper website based here in Washington DC and it published lots of news about that was very pro-russian uh very critical of the United States and it was used to leak Sensational stories about U us politicians the overall aim of the website was to convince uh us audiences or people who in other parts of of the world who read us media uh that um certain things about us politicians and the US government were not true so it presented the US in a very negative light and presented Russia in a very very positive light and so DC weekly.org uh some investigative journalists dug into it and they identified uh I think originally from an old who's record I think that was likely linked to a guy called John Mark Dugan now John Dugan was a sheriff's deputy he who lived in Florida originally and uh in 2016 he fell out with his Employers in the uh in the police department where he worked in Florida and he set up a series of fake news websites to publish um defamatory information about uh other law enforcement officers about judges in Florida and so on now if you work in law enforcement you know that's going to end very badly for you if you do that kind of thing and just before he could be prosecuted he fled to Russia um to avoid prosecution in the United States and he's still based there so journalists um said hey this DC weekly website that's publishing all this information about the US government all this Russian disinformation uh we think it's yours John Dugan he said no um he obviously clearly denied it but if we dig into the security certificates that he obtained for DC weekly.org we can see U there's quite there's an overlap between his domain DC weekly.org and some others as well remember security certificates you have to prove ownership of a domain and once you've published it you can't go back and change uh you can't you can't take away what has already been made public uh through certificate transparency logs so we can go to a website called c. and we can look at every single security certificate that has ever been issued for DC weekly.org and if we go back to the very early days of the website uh we could see that when he obtained a certificate or the owner of DC weekly obtained a certificate he obtained one for DC weekly.org the same certificate also covered clear story news which is another disinformation website and it also covered another another um subdomain on another website called falconite Tech all these websites unfortunately or fortunately uh now gone offline but what that record there shows us is that the same person owned or controlled all of these websites and when we went to the Falcon ey. Tech website uh you could see it was the it was the personal business website of John Dugan he ran even though he was based in Russia he was offering sort remote security monitoring Ser monitoring services for people in Florida where he used to work uh but what that record tells us is that all three of those domains are all owned and controlled by the same person so his claim clim uh that this is nothing to do with me well actually the security certificate records show that if you own Falcon ey. Tech which is your openly your own personal security business uh you also must own and control DC weekly.org as well so uh who is records don't help you out this kind of thing uh hosting history doesn't help you this kind of thing but that you can't lie uh to obtain a security certificate so it provides a very strong signal uh that the same person is linked to this to to all the this whole collection of different websites another really useful example of how we can use security certificates is to find hidden infrastructure uh by hidden infrastructure I mean infrastructure that is still an essential part of an organization's online footprint but perhaps is not designed to be accessible to the public so it might be um areas for internal Administration uh and so on it doesn't necessarily mean that they're doing anything anything wrong or anything suspicious but if I own a very large organization and I have lots of different parts to my company's domain um I probably need to obtain security certificate separately for different parts of that infrastructure so one of the very useful things that security certificates do is they help us to map the subdomains of a comp of an organization's sort of Main website their main domain uh and we get a much bigger uh picture of what their online infrastructure looks like and very often uh threat actors make mistakes or by they reveal things in their subdomains uh that isn't aren't obvious any other way uh and I'm going to show you now an example of a Russian state-run media operation uh that exposed itself in this way so the backstory so after Russia invaded Ukraine which is uh three years ago today as we heard this morning a lot of Russian State media organizations were sanctioned uh and they were banned from Western social media platforms and yeah and a lot of their websites became inaccessible in other parts of the world uh which was a problem for Russia because they to try and steer the world towards their point of view they seek to dominate the information space so push out Western media and uh inject their own narratives their own claims into that space and they use their State media Assets in order to do that so this was a bit of a blow to Russia um when sites like Russ or rt.com and and others uh were no longer accessible in the west so one of the ways they tried to adapt to this was to try they would try and set up new domain new brands new domains to try and disguise uh their state operations they didn't they didn't give up the ban didn't stop them they just had to Rebrand themselves and they set up quite a few of new brands new domains new websites in order to try and do that now the internet has rules and one of those rules you have to follow is if you want to set up a website you must have a security certificate because if you don't uh people's browsers will not trust the website and they won't be able to visit it they'll just get a warning in their browser the other thing uh that will happen is uh if you don't have a valid security certificate Google and other search engines will rank you in their search listings so you need to have a security certificate doesn't matter if you're a nation St nation state trying to hide your operations you still got to have one so in June 2023 this new website popped up and it had various other social media assets on on various platforms uh called vii. videoo brand new Enterprise and it was didn't say it was part of the Russian State um it didn't admit that it had any links to previous known Uh Russian State media operations but they set up this website and is used it's used to um push video content that tend to support Russian messaging not in the west so much um but in other parts of the world in in the global South so we want to investigate this website and say well where did it come from it's very well resourced it's pushing Russian propaganda how can how can we dig into it one of the first things we can do we can go to se. sh and look at the history of every single security certificate ever issued for that website or for that domain add any of the subdomains uh little side note here where security certificates so hard to say that word um where where they are also useful they will tell you when a website first went online as well or at least give you a rough indication and we can see the very first security certificate for this particular domain was uh set up in June 2023 now look through the security certificate history we not interested so much in the main domain but in February last year so in fact one year ago this new subdomain popped up called areu election 2024 vii. videoo now the Russians have elections U of of of a kind uh and they had one to essentially reelect President Putin this time last year and this subdomain popped up now this is super interesting to me because if you Google for that term Ru election 2024 vor. videoo you won't find it indexed in any search engine um it's not linked to internally from the viori website um the only way that as far as I know to have discovered this particular subdomain was through looking at the certificate history so being nosy uh went to visit this particular website which I think is still there and it offers uh front row views for the announcement of the results results of the Russian election so you could pay €450 uh that gives you a front row seat for the announcement of the Russian election results uh you have can have a fully equipped and staffed lived standup position overlooking Red Square for when they they reann nowc the election of President Putin uh they also offer to facilitate direct interviews uh with Russian politicians and government officials now put your analyst hat on for a moment and think on balance of probability making your decisions um How likely is it that this bori who are offering these services are linked to the Russian State based on what this information the indicators like if this was a genuine independent New Media brand very very unlikely that they would be able to offer direct access to to Russian politicians so uh this was a big red flag to me and helped me I thought from this well it does tend to support the idea that viori is a disguised new uh it's the same old Russian State media under a new brand uh to beat sanctions and to beat platform evation there's something else we can do with the security certificates as well and look at some of the other subdomains so I went back to sir do uhsh and looked for some of the other certificates they'd obtained and I noticed they have this really strange way of naming their subdomains so you see here they're all like a cdn2 nc4 cdn2 en nc3 bi.v videoo and so on and so on and so on all okay admins make their own decisions about how they name their infrastructure but I thought this is kind of weird um do any other organizations use this same weird subdomain format so I went to security Trails which has a huge historic records of DNS records and I did a wild card search uh for this same term C d2c uh and guess what in the whole history of DNS records covering the entire history of the internet going back to the the early part of this Century only two organizations in the entire history of the internet use that same naming format uh for their subdomains one was vi video this new suspicious Russian media operation the other organization uh was one called ruply now ruply uh ruple dotv um is overtly a Russian State media operation and they were sanctioned by the US in in September last year um but rutley and uh viori are the only two organizations in the entire world ever uh that use that same subd name naming format which is a very strong link that we can find through the certificates they also guess what they these U these CDN subdomains also Al share uh the same single IP address so thinking about now about your attribution it seemed we thought they're likely to be linked to the Russian State because they were facilitating interviews with Russian politicians they're also sharing the exact same internet infrastructure as a known Uh Russian uh State media entity as well and then we dig into the people and we find this very convenient Behavior where we go on LinkedIn and a lot of people who used to work for suddenly Now find themselves they're now employees of bori video based in Dubai so on the balance of probabilities yeah highly likely uh that viori is one of these disguised subversive Russian State media brands that allows them even when they've shut down on one platform uh to Rebrand themselves uh and move to others to continue their their propaganda operations take away from this we would not have been able to discover this information through Googling we wouldn't have been able to discover it through through examining the website uh the security certificates are that's the resource that points us to all these other parts of their infrastructure that allow us to make uh these these um these connections so ruly and most of the other main Russian social Med um Russian State media Brands were sanctioned by the US last year viori somehow missed the chop so it's not hasn't been sanctioned um but we can see very clearly um that the certificates don't lie okay final section now uh something a little bit more complicated uh something a little bit more novel from some research I did earlier this year which is a completely different use of security certificates and this is to find the real IP address of a threat actor website that hides behind Cloud flare now if you've done website investigations uh for any time period before you will have inevitably sometimes come up against um come up against Cloud flare and I for the record I'm a big fan of cloudfare they do some fantastic stuff to help protect websites from things like dos and so on uh but for investigators Cloud flare can be very very frustrating because when you want to know where a particular website is hosted uh you can you try and visit it and oh sorry you'll look at the DNS records and it'll just say cloud flare cloud Cloud play very very difficult to know where a website behind Cloud play is really hosted and there are a few techniques for dealing with that that i' I've covered in uh sans's talks previously uh but this is a new method that I tried uh earlier this year and uh it uh it served me very well so a little bit the theory first of all so if you if you set up a website so if I'm a a threat actor and I want to set up my propaganda website to influence an election for example I set up my website here on the origin server so the real IP address the real server the origin server and I connect to Cloud flares infrastructure uh somewhere here in the middle and then any users who come to visit my website they'd never connect to my origin server they will only ever connect to Cloud flare so they don't ever see the real IP address and uh they don't ever connect directly to the original website uh they just connect to Cloud players's Edge which works very well it's just frustrating as an investigator so what that means very often is if I want to investigate a website that uses Cloud flare and I want to look at their security certificates what I'm actually going to see when I look at um sir Dosh or when I um visit the website and inspect the certificate directly I'm going to see the security certificate not on An Origin server but I'm going to see Cloud flares certificate which can make it very hard but what cloud flare also recommend when you set up your website is to make is that you also obtain a separate security certificate for your origin server to encrypt the connection between your server and cloudflare you don't have to do that um but it's recommended and most sites do so bear with me on this one but what that means is if somehow I can find the the or server security certificate I can verify the real IP address of the website where that is small problem uh there are 4.3 billion possible IP version 4 addresses even if we ex remove some of those um but I kind of like that as an investigator because it means that our subject is cornered they've got nowhere else to go there's only 4.3 billion possibilities like they they've got to be there somewhere right they are there somewhere and eventually we'll find like a needle in a Hy stack we'll find the right one I'll explain how a little bit more but I'm just going to cover how this technology works very very briefly so when you connect to a website you put the address bar in your in your so you put the address in your browser bar hit enter um very very quickly your your browser performs something called a TLS handshake do it establishes an encrypted connection between your browser and that server it only takes a fraction of a second you don't even notice it uh but that TLS handshake takes place before any traffic traffic goes back and forth between your browser and the Machine which that kind of works okay but then we have another problem when I connect to a server there might be five different websites on on that particular server there might be a thousand there might be 10,000 so when I connect to when my browser connects to a server at a given IP address how does it make sure that I make perform an encrypt a connection with the with the re with the right domain because there's thousands to choose from how do I actually do that and the way the internet handles this problem is through something called server name indication I'll show you how that works and how we can exploit that uh in just a moment so server name indication Works something like this so iio there's a server somewhere which has three websites on it it has example.com it has test.com and has company.com so when I connect to that server how do I make sure that I form an En crypted connection with the right one well server name indication means when my browser tries to uh establish that encrypted connection it will name the domain that it wants a certificate for so when I connect to this server here if I want to connect to example.com my browser will use uh will specify in um example .c as the domain that it wants to connect to and then if there is a valid certificate for example.com on that server uh the server will will complete the connection and we and we connect to the website if that certificate is not there it will say and it will just give me say I haven't got a certificate for example.com and it will just present me with a certificate for for the default domain if there is one um so we can think of this as essentially knocking on lots of different doors uh asking if someone is at home so if I want to visit Matt for example and I don't know where he lives I could knock on the door of every single house in his neighborhood say is Matt there no is Matt there no is Matt there no but if Matt is there eventually I'll find one house where he opens the door and I find him um so that's how TLS works and this is how we can take advantage of it so there is a very complex wellestablished very well resourced Russian uh disinformation operation called um what the that we refer to as the prava network it's a vast network of websites that Target um pretty much every single country in the world but their English language operation is called PR thean they target the UK and the US and I want to know where this is really hosted I suspect it's Russian because of the content but uh following on from what Andrew said this morning we want to do we want to be good at attribution uh we want to try and show our working not just guess so we want to try and establish uh where this website is really hosted problem with that cloud flare they use cloud flare so we can't easily find uh their real IP address of the website bit of a problem so to go back to my theory um finding a needle in a hay stack I know somewhere on one of the world's 4.3 billion IP version 4 addresses one of those IP addresses the Pravda website is there don't know which one um but I can I can have a guess but um I can find it eventually given enough time because finding a needle in a hay stack is actually pretty easy um if you were looking at me thinking what of course it's not easy well if you think if you have a huge hay stack of 4.3 billion bits of hay and one needle you would find it eventually if you tried removed every single bit of uh bit of hay you would eventually be left with a needle very boring uh very time consuming but conceptually not not too tricky so I thought if I can perform as a TLS handshake with millions of different servers and I ask for the domain Prav deen.com eventually I will be wrong every single time until eventually uh the origin Sero will present itself to me when I perform a TLS handshake um so I thought well that's that's kind of fun as an idea how could I actually do this so the great thing is uh there is a free tool available uh called zg grab 2 uh if any of you have ever used census um it's it's the same family of tools that they use to map the internet and scale uh and zg grab 2 you can provide it with a range of IP addresses uh and it will go and perform a TLS handshake for you uh and and announce the results to you so um I said that you could use zg grab 2 to connect to every single possible IP address to for the domain you're looking from I don't recommend that you go for the whole world's 4.3 billion addresses all at once you have to think a little bit smart um so if you know for example that a threat actor always likes to use the same um hosting provider you can narrow the range down to just a few million IP addresses or if they have other infrastructure that you think is probably part of the same operation you can start there um rather than going for the whole internet at once now there obviously I I assumed it was Russian so I grabbed the IP ranges of different Russian hosting providers prior a site from bgp doio uh and I tested this concept out to give you an idea of how long this takes uh to scan one of the one of the asns was about two and a half million IP addresses that took about two hours to scan that so it's pry it's very very quick um and I can grap through the results very quickly and see if I got a match or not so does it does doing the whole internet would take a few days but actually that's not that long a time in the scale of things so this is how I run zg grab so this is the command um you run zg grab on a on a server don't run this from your own machine um but you can run run Z grab on a virtual server somewhere and I tried a few different Russian hosting providers and I got no matches back but um I there was one of the Smalling host smaller hosting providers a company called logal very very small Russian hosting provider their whole um IP range is only about 7,000 different IP addresses so very very small as far as a hosting provider goes and the command I ran went something like this so I run zg grab 2 in TLS mode so it's going to do that handshake I give it the input file which is the list of IP addresses or IP ranges that they use the output file which is going to be my results Json and then this option here server name pra.com that's my that's the Sni um part of the the handshake so I'm asking every single IP address in that range to say have you got certificate for this domain and of course no no no no no no no yes and eventually after only a few minutes because this is a very small number of IP addresses this IP address here 178 21585 comes back and says hey yes I do have a certificate for proud deen.com on my server and of course this is this shows to me then that this is the origin server this is not the cloud flare certificate this is not the cloud flare IP address I've been able to connect eventually after a lot of trial and error um to another IP address and grab um the origin certificate from that particular IP address so now very confident uh that I found where this infrastructure is and I'm going to be able to act attribute the site not only based on the content but also uh based on the infrastructure as well now because we want to be diligent ENT investigators we don't just want to guess we want to verify and test that our findings are accurate you can reproduce this Pro um process on a per server basis U using a tool called SS SS ssli uh which is it's a python tool if you've used Cali Linux it's already built into Cali but I can reproduce this process uh using the command that you see there uh by connecting directly to the IP address um you can see I specified it in the command there then using the Sni option for the domain that I actually want if that domain is there it will it will presentent me with genuine certificate for that domain it did in this case you can see the copy of it there you see the um shaan fingerprint of the certificate I can compare this to certificate transpar parency logs and I know I know then that I've got the right one and I've found it and that I'm able to verify it and then I'm able to put into my report that yes this Russian content website is definitely hosted at this IP address and that's all through uh all through ultimately through taking advantage of the way security certificate works okay well that's going to uh that's pretty much the end of my time um so any if you have any questions about what we covered or you want to discuss the techniques in any more depth um please post a question in the slack uh and I'll answer them I know it'll be a question about don't you get caught when you blast thousands of IP addresses at once the answer for that is no uh very often because most web servers are not checking for failed TLS handshakes is a short answer I can discuss that with you in more depth um and tomorrow I'll be running a workshop where we'll dive into way more depth into some other techniques for looking for these kind of websites thank you very much everybody thank you than you Back To Top