DATA ANALYTICS FOR CYBER Cheat Sheet by depasinre2

Premis­e/p­roblem JS can cause:­bitcoin mining, abuse browser vulner­abi­lities

JSTAP Principle： Perform static analysis with abstract syntax trees and random forests

Static analysis - we don’t run the code at all, Reverse analysis of the code

Dynamic Analysis pitfalls - 1. Easy to detect you are in a debugger, VM, or running Anti- virus – Query registry – IsDebu­gge­rPr­esent – VM specific instru­ctions. 2. Do long delay in hopes simulator will give up and go away

Program Dependence Graph – Includes data and control depend­encies

N-gram - simple way to analyze token sequences

- Depth-­first pre-order traversal of AST

- Similar for PDG, consid­ering data flow

- 4 is the best value.

- if ngram in both (benh and malc), throw ngram away

JSTAP Classifier Training • Select 10,000 malicious and benign randomly for training – Additional 5,000 of each for validation • Repeat 5 times and average detection results

Evasion techniques - Add more benign features • Copy malicious into larger benign file

Abstract execution - A Technique for Effici­ently Tracing Programs. In a dynamic analysis, it has Emulator, Extremely avstract OS and paths, Less precise than virtua­liz­ation or emulation

Techniques for finding Phish:

- User-I­nte­rfa­ce-­based: Eg provide custom image per user, Password manager (Only provides password to certain domains)

–- CANTINA: compute term freque­ncy­-in­verse document frequency for terms, then Google a few terms to see if current website is a top result – B-APT: Bayesian based on tokens from DOM

Textual and visual classi­fic­ation: text classi­fiers work by examining text within a page to detect whether certain words are more likely in a fraudulent page or not. Image classi­fiers transform webpage to images and then compares similarity to genuine webpages.

Overall framework 1. Train text and image classi­fier, collect similarity measur­ements for different classi­fiers 2. Partition similarity into sub-in­tervals 3. Estimate probs for text classifier 4. Estimate probs for image classifier 5. Classify each test image 6. If different from two classi­fiers, calculate decision factor 7. Return final classi­fic­ation

access­ibility: publicly available;comple­teness: encompass all the breadth within phishing; consis­tency : range and variance of dataset to make sure data won't be substa­ntively changing; integrity: data and labels is correct, non-co­rru­pted; Validity: data is properly repres­ent­ative; interp­ret­ability : data is unders­tan­dable; Timeli­ness: data is updated or still valid today and future

boosting classifier is random forests build each tree indepe­ndently while gradient boosting builds one tree at a time. This additive model (ensemble) works in a forward stage-wise manner, introd­ucing a weak learner to improve the shortc­omings of existing weak learners.

Criminal accounts tend to be socially connected, Maybe less discri­min­ating in who they follow – Maybe intent­ional

K-anon­ymity - Publisher decides which attributes public­/pr­ivate – Public are “quasi­-id­ent­ifiers” • Every quasi-­ide­ntifier tuple appears in at least k records in anonymized DB

1. Compute score(aux, r’) for each r’ in sample 2. Apply matching criteria 3. Output record or probab­ility distri­bution for records

How bystander detection could improve privacy: this can stop bystanders from being recorded without knowing or let them know. Self-c­entered photos can put bystanders in awkward situat­ions, poor posture, or reveal inform­ation they don't want on record,

How to get < 10% unicity • Remove all info pertaining to clients and website visits • Coarsen time to at least hours

challenges in defining “fake news” - apart from validity of inform­ation, is it satire, actual misinf­orm­ation, intended for deception, clickbait, rumor etc.

fact extrac­tion: redund­anc­y(D­ona­ldj­ohn­trump vs donald­-tr­ump), timeli­nes­s(B­ritain, joinIn, Europe­anU­nion), conflict, unreli­abi­lit­y(T­heO­nion), incomp­let­ene­ss(May need to infer if something is missing)

Explain how textual and visual analysis may help with fake news detection

- visual content can clarity, coherence, similarity distri­bution, diversity and clustering score.

mixed code - Use of different languages, symbols, scripts, shapes to avoid detection. Text on Document – Defined from standard alphabetic characters • Text in Visual Media – Text in pictures • Text as Art Form** – Use symbols not part of the alphabet to depict a simple code

bi-clique - bipartite graph where every vertex of first is connected to every vertex of second

Find temporal cohesion, And textual cohesion, And created weighted sum, For an article, average its score in all bi- cliques,

Spread labels if – Part of same bi-cliques – Have a lot of common users – Are textually similar,

Dark Web: (Tor) overlay networks that use the Internet but require specific software, config­ura­tions, or author­ization to access -Behind password logins – Encrypted – Not linked – Tor Hidden Servcies

Tor browsing: use many(3) different machine to create onion networks. Each connection is encripted beside of the exit.The exit will appear to be browsing.

1.pick introd­uction points to build encrpted tunnels 2. announce the service into db. 3. User get back to 3 introd­uction points and create rendezvous points (3 steps from) and 4. send msg to intro point. 5. now the rend point is 6 hops away from intro.

socially detrim­ental uses of Tor and anonymous browsing: can be used as a harbor for illegal/ illicit things

how resear­chers have crawled the dark web: first get access by identi­fying dark web forms. Then get data thru anon access, then process and identify relati­ons­hips/ link data sources etc. then visual­ization and reports

Inform­ation gain - reduction of entropy gained by knowing feature x: IG(y|x) = H(y) – H(y|x)

insider threat and accidental insider threat: threats from within (emplo­yees, associ­ates) weak passwords, unlocked devices intent­ional can be injecting rogue software

Advantage of a hidden Markov model over an SVM for classi­fying command sequences: Markov model creates probab­ility of each transi­tion; this can easily grow very big; pick a K that is small; svm can be very accurate but it does not address concept drift very well