Phishing campaigns using AI to avoid detection

[OVERKILL]

Pretty sure everybody saw this one coming
quote from the article:

Microsoft warns that a recent phishing campaign used AI technology to obfuscate its payload and evade security filters.

"Appearing to be aided by a large language model (LLM), the activity obfuscated its behavior within an SVG file, leveraging business terminology and a synthetic structure to disguise its malicious intent," the researchers write.

"In analyzing the malicious file, Microsoft Security Copilot assessed that the code was 'not something a human would typically write from scratch due to its complexity, verbosity, and lack of practical utility.'"

The attackers used a compromised small business email account to send the phishing emails, which posed as file-sharing notifications. If a user opened the attached file, they would be redirected to a webpage designed to steal their credentials.

Microsoft notes, "The attackers employed a self-addressed email tactic, where the sender and recipient addresses matched, and actual targets were hidden in the BCC field, which is done to attempt to bypass basic detection heuristics."

The researchers warn that this campaign is part of a larger trend of threat actors using AI tools to assist in "Like many transformative technologies, AI is being adopted by both defenders and cybercriminals," Microsoft says.

One of the easy "tells" from mainstream phishing campaigns have been bad grammar, incorrect spelling...etc. Because they are often orchestrated by non-English speakers (at least as their primary language). With access to AI writing tools, this has been improved upon and now they are harder to spot by these metrics alone, but Microsoft and other vendors have been tweaking their threat detection software to catch them, following the links to determine if they are malicious and implementing things like MS's "Safe Links" to avoid malicious URL's.

With this latest twist, using an LLM to hide the malicious nature of the content within a file, the bad actors have again stepped up their game and vendors are playing catch-up. As AI improves, we will continue to see more, and more complex, variations of this sort of strategy in this seemingly endless game of cat and mouse.

https://blog.knowbe4.com/new-phishing-campaign-uses-ai-tools-to-evade-detection

 

[JeffKeryk]

While phishing campaigns and bad actors are nothing new, AI is a powerful tool.
My last job was for Blue Coat / Symantec Cyber Security. This activity costs us A LOT!

Cat and mouse is a good analogy. How do you fix something until you recognize it?
There will be blood...

 

[Speedbird001]

... One of the easy "tells" from mainstream phishing campaigns have been bad grammar, incorrect spelling...etc...

I see phishing attempts several times a week. One "tell" is the use of British English phrases and words vs US English... Large % of English teachers overseas are from the UK, thus even when well schooled the scammers unknowingly let the UK slip thru.

 

[Vercingetorix]

I read clock about a fellow who shoe has started throwing pizza irrelevant words Snoopy out so as to grass throw the algorithms off.

 

[Y_K]

Not an AI one, yet a witty phishing trick:

https://www.netcraft.com/blog/down-the-hiragana-hole-uncovering-a-new-wave-of-lookalike-domains

 

[alarmguy]

I never open mail with attachments unless I am expecting it from a known source. Like anything anyone can make a mistake but in my pre-retirement job I was trained and tested by my company on a regular basis. You could not even click on an unknown email. If you failed you went back to "class" ... my wife's company currently tests her regularly as they handle constant "art work" for orders/

Here is my question. Most all times I will not even open the e-mail or text message.
Im almost certain I know the answer but do you really need to download a file anymore? Cant the link itself be the redirect to a rogue website and install something malicious?

Even if something looks convincing to me if I dont know what exactly it is. I think to myself if it is important I will get a letter in the mail.
One recent experience highlighting this was a trip to a local urgent care center. A couple weeks later I kept getting text messages to pay a co-pay by clicking a link. This text was coming from one of those random numbered texts, not a phone either.

Anyway I ignored the texts, quite a few. One time when I figured out it most likely was legit. I responded to mail me a bill if you want this paid. Not sure if they got the text or not but I got the bill then paid it online. I feel in in hurried society we many times feel the need to open something up right away. If it's a bill or security issue coming in an email or text. Have them send it USPS OR find a phone number on your own to contact them. Such as we already know our doctors, loan companies, credit card companies by ourselves, dont trust a number in an email or text. However this does not answer my question in bold type above.

 

[Pablo]

That's it. Out of the net now. Shower and dry off, you're done AI.

Point: You lay around with dogs with fleas, you will get fleas. Avoid anything that you have not initiated yourself and even then double check.

 

[andrew_j]

All our phishing test are from external emails so I simply filter to a bucket called External Spam and typically click Report Phishing or delete to stay compliant.

 

[2005jettatdi]

As a tax preparer I am required to have a security plan. One of my strict rules is I will not open and attachment or link in an email unless there is a personal message in the body of the email. Some people cannot understand the reason for this and think I am just being picky.

 

[Brons2]

Not an AI one, yet a witty phishing trick:

https://www.netcraft.com/blog/down-the-hiragana-hole-uncovering-a-new-wave-of-lookalike-domains

This isn't really new, but has ramped up in recent years.

It is why most organizations with reasonably competent security staff block all newly created domains in their web filters. We don't allow any newly created domain for 30 days. Occasionally causes some blowback with some shadow IT created website, but, we just manually unblock it if that happens.

 

[Owen Lucas]

AI will catch-up. At least Google will IMHO. There will always be emails that get through, the one-I am worried about are those that can spoof the domain name of a legitimate company.

 

[Pew]

Pretty much any email that contains the phrase "can you kindly please....." is automatically spam. The threat actors are gonna have to find a new catch phrase.

 

[OVERKILL]

AI will catch-up. At least Google will IMHO. There will always be emails that get through, the one-I am worried about are those that can spoof the domain name of a legitimate company.

It's not the spoofing that should have you worried, any properly configured mail system should generally reject, or at least quarantine mail that doesn't pass DKIM, SPF and DMARC:

It's the hijacking of legitimate accounts on domains that pass the above that are typically the biggest problem.

 

[Blagoje]

I see phishing attempts several times a week. One "tell" is the use of British English phrases and words vs US English... Large % of English teachers overseas are from the UK, thus even when well schooled the scammers unknowingly let the UK slip thru.

I'm doing more with the Russian language as of late and AI is not particularly good at translating to and from English and Russian. There is plenty of nuance in both languages that one cannot easily synthesize from simple information consumption and therefore it is very easy to spot glaring (or even not so glaring) errors if you actually have real world human experience.

 

[ripcord]

Somebody (or everybody) out there has built or is building the world's most sophisticated penetration testing AI. WWIII will be a cyber war. Every website, every application, every OS, every encryption algorithm, every system analyzed, attacked and exploited.

All your security are belong to us.

 

[PandaBear]

Bad human can be just as dangerous as bad AI. I just refuse to open any attachment unless it is from within my company on work email and in my personal email I only go to the company's official website and search for the equivalent link myself. Also if someone call me say they are from who and where I always tell them I will call directly from the official company phone line instead of believing in what they say, I will call and get them from the company's official number extension instead.

You can't trust anything human or AI if they initiate it. You always have to go find the link yourself from official address, website, phone number etc.

 

[PandaBear]

I see phishing attempts several times a week. One "tell" is the use of British English phrases and words vs US English... Large % of English teachers overseas are from the UK, thus even when well schooled the scammers unknowingly let the UK slip thru.

Indian.

 

[Pablo]

People still getting their PC's "taken over"! My wife got one yesterday from a friend of a friend.......hint hint. (asking to buy a $500 Amazon gift card because her funds were on hold or some garbage)

So she CALLED the friend. Yep. hacked. Sent emails to every one of her contacts....

 

[97prizm]

While phishing campaigns and bad actors are nothing new, AI is a powerful tool.
My last job was for Blue Coat / Symantec Cyber Security. This activity costs us A LOT!

Cat and mouse is a good analogy. How do you fix something until you recognize it?
There will be blood...

Sounds like auto theft over the years. Used to be a coat hanger and screwdriver and the car was yours. Then lock guards, then specialty cut keys, then fobs and so on.

 

[JeffKeryk]

Sounds like auto theft over the years. Used to be a coat hanger and screwdriver and the car was yours. Then lock guards, then specialty cut keys, then fobs and so on.

Sure. The good news car theft is far more diffucult in modern high tech vehicles, typically EVs, due to their unique, high-tech security features.
You can guess which is the hardest... Just don't leave your cell in the car!