Infusing AI with the Human Element

How CPRG’s AI programs approximate real-world poker tactics

Poker’s move into the digital realm has been very beneficial for the gaming industry. Even with the card game’s popularity already reaching new heights due to the exposure provided by the 2003 World Series of Poker TV broadcast, online poker was able to further push that popularity up to 11 thanks to its wider reach and more accessible nature.

A lot of poker pros actually got their start by plying their skills first in virtual poker rooms. In fact, established superstars like Patrik Antonius and Phil Ivey still make their respective presences felt in online matches. These indicate that just like their real-world counterparts, online poker games provide just as rich a tactical gameplay experience. Online gaming operator bwin.party’s PartyPoker brand even provides strategies on its blog page that cater specifically to digital matches. Furthermore, its YouTube channel has additional online-specific tutorials hosted by Kara Scott, herself a pro player.

The question, of course, is how the AI of virtual poker compares to good old-fashioned human thinking; and if the University of Alberta’s Computer Poker Research Group (CPRG) has anything to say about it, the current state of poker AI is highly comparable, if not 100% emulated. And the best part is, it’s constantly evolving.

Most video gamers know the CPRG as the team behind Poki AI, the highly acclaimed artificial intelligence program used to power the video game Stacked with Daniel Negreanu. The program’s claim to fame is its ability to switch tactics on the fly instead of sticking to one pre-programmed set of instructions, allowing for more dynamic gameplay akin to actual poker sessions.

As amazing as Poki is, a more notable achievement for the CPRG is its series of AI collectively called Hyperborean. These AI codes have proven their mettle in the Annual Computer Poker Competition. As CPRG researcher Richard Gibson explains, the Hyperborean programs operate on a system of tactics that take Nash equilibrium into heavy consideration.

To better illustrate this, Gibson differentiates Hyperborean from the chess AI Deep Blue. The latter, he says, relies primarily on a technique called the alpha-beta search, where the aim is to go for that one move statistically proven to have the best outcome given the specific arrangement of the remaining pieces.

On the other hand, since poker has a lot more unknown variables to consider, Hyperborean sees the best strategy as that of a more defensive bent, where the assumption is no one player (human or otherwise) has anything to gain without considering the strategy of the others. Therefore, until a game goes into its later stages when more constants are introduced, the best bet (pun intended) is to hold back until more cards are dealt.

That, of course, is a highly simplified explanation of the obviously more complex programming coded into CPRG’s poker AIs. The point remains, though, that the way these programs approximate human thinking is getting more and more fully realized.

Things That Might Indicate Your Computer Is Infected

Despite the title, it’s important to note that if your computer is infected you may not have any indication of that infection. Much malware leaves no visible evidence of its presence. But, some does and below are a few of the things that should cause you to investigate further…

  • Your browser takes you to odd places. If a bookmark that has been directing you to your favorite site suddenly starts to direct you to another site that maybe tries to sell you something then the bookmark might have been hijacked by malware. Similarly, if you type in a URL but get taken to a completely different site address that should raise your suspicions.
  • You try to access your anti-virus software but can’t. A number of malware beasts will attempt to turn off your anti-malware protections (e.g., anti-virus software, adware detectors, etc.). Often they are successful and sometimes will even deactivate any shortcuts to the program(s). If you’ve tried to access your anti-malware software and can’t, that’s another reason your suspicions should be raised.
  • You start seeing ads but your browser is not active. If ads start to pop-up on your system without any reason you should be very suspicious. Even worse, if these pop-ups say your computer is infected and attempt to sell you a solution, don’t wait. Immediately find out what sort of malware you have and get rid of it. That’s pretty much a dead giveaway malware is afoot.
  • Your social media sites start showing posts from you that you didn’t write. Fake postings on social sites are a strong indication your computer is infected. Their purpose is to get your friends and followers to click on links in the post so that your friends and followers might become the next victim. Do them a favor and investigate your system for malware.
  • Systems tools are unavailable. One of the ways to investigate what’s running on your system is to use the Task Manager and other related system tools to see what programs and processes are running. It’s not a foolproof method because many legitimate processes have really odd names. But, if you can’t open these tools at all via an administrator account then you should seriously look into the possibility your computer is infected.
  • Friends start to call you about being stranded on your trip to [insert any far-away place]. One of the payloads in malware is to send E-mails to your address list and other mail addresses found on your system. A popular messages appears to be from you saying that you are stranded in some exotic place and you are asking them to send money to help you get back home. The instructions are to wire it to some location. The problem is you are not there and once you wire money from your bank account you can’t usually get it back. Calls like these from correspondents should be a clear sign you have an infection.

There are probably other subtle signals you could look for (e.g., system runs very slow, the internet connection or hard disk seems particularly busy for no apparent reason, etc.) but the ones above or indications similar to those are pretty common signs of an infection. Get some anti-virus software and check your system as soon as any of these show up. Better, have some running before the infection to help keep your system clean.

Random Thoughts

This is the start of the CKnow blog. It will be a random entry blog of items of interest to the CKnow Website owner and posting will depend on when those items appear and what’s happening in the owner’s personal life that controls how much time can be spent working on the site.

Sorry to be so general but I’m currently working on my third full career and have many new things I would like to explore in addition to that.

I hope you enjoy and find useful those comments that make it here and the rest of the Computer Knowledge site.

Google’s Server Setup

CNET News has posted a story about how Google manages their server farm(s). This once secret information reveals a company who, despite their revenue, is looking after all the pennies and trying to do so in a way that helps the environment as well.

They design their own servers and use 12v DC power supplies with power conversion on the motherboard so no AC to DC conversion is necessary on the server itself. To top that off, no UPS is involved; each server has its own 12v battery attached to it for backup power purposes.

Everything is housed in standard shipping containers (like those you see on cargo ships) which can be moved quite easily if necessary. [This is not unique to Google but perhaps it was when they started doing it in 2005.]

Quite clever actually. And, very efficient.

Full story here: http://news.cnet.com/8301-1001_3-10209580-92.htmlWeb Link

Added: And here is a link to the original designs used to test the search methods…

http://infolab.stanford.edu/pub/voy/museum/pictures/display/0-4-Google.htmWeb Link

Cyber Attacks Getting Worse?

[Original post 4/10/2009] Increasing news reports are indicating that cyber attacks are both increasing and getting increasingly bad. A number of reports highlight an attack on the US power grid; some even indicating that malware has been planted so that foreign powers can wreak havoc on cyber command. China and Russia (among others) have been implicated.

But, no proof has been offered (at least not in any of the reports I’ve seen). It would be nice to have some details instead of the “sky is falling” reports that seem to make today’s headlines for what are called news stories.

Understand, I don’t doubt the grid is vulnerable and that probes have been performed and that the power companies have no clue this has happened until told by others. Malware insertion, however, is another level and major claims require major proof which has not yet been offered.

But, if someone did install malware and the PTB* know about it I certainly hope they’ve taken the next obvious step and removed it; not to mention blocking the hole it crawled in through. We don’t know because that part has not made the news.

China, not surprisingly says: “The incident of attacks on the U.S. electrical grid from China and Russia simply does not exist,” (Chinese foreign ministry spokeswoman Jiang Yu).

References (among many others):

*PTB = Powers That Be

Twitter Worm(s)

Twitter and other services need to address security. You, the user, need to as well.

Yet another Twitter cross-site scripting worm is making its rounds as I type this. It’s a variation on the previous worm that Michael Mooney (Mikeey) confessed to writing not long ago.

Cross-site scripting is a well known security problem that basically allows one page to load things into a user’s browser from one site and then inject code into a page from a different site. The code can be simple HTML or complex scripting. The user at the browser may not even know something is going on!

While it’s easy for a developer to miss vulnerable code, it’s not impossible to check for the holes that allow cross-site scripting. Problem is many do not in their rush to get product out the door.

Twitter had such immediate success it’s very likely they rushed code out the door to keep up with growing demand and thus opened themselves up to the vulnerability. That means that now, however, they have to do that checking after the fact and what vulnerabilities are still there will certainly be found and exploited. Too bad.

The message(s) the new worm leaves say it too well: “Twitter, this sucks! Fix your coding.” “Twitter Security Team Really? You need to be fired.” and “Horrible Coding!”

Of course, the user is not off the hook. Safe browsing is important. This is why I often recommend that people use FirefoxWeb Link with the NoScript add-inWeb Link running in the background. That completely blocks cross-site scripting and other exploits from running unless you tell them to. Makes some sites look bad but worth the effort when browsing around the wild-west Internet.

For more on cross-site scripting see WikipediaWeb Link.

Les Misérables

[Original post 5/6/2009] Les Misérables as put on by the California Central Coast-based PCPA organization is an outstanding presentation.

PCPAWeb Link (Pacific Conservatory of the Performing Arts) is affiliated with Allan Hancock College in Santa Maria, California — in the wine belt of the California Central Coast. It is unique in that it is the only training program of its kind in the country offered by a community college. They perform out of two venues: The Marian Theatre at the college in Santa Maria, and the Solvang Festival Theater, an outdoor theater in the city of Solvang about a half hour south of Santa Maria.

Of note, all performances of the major offerings of the year are in a smallish theater in the round with a raised central stage and tiered seating around about 270-degrees of the circle. The rest of the circle is attached to the stage and is where most scene changes and the like take place. The flexibility shown in the use of this small space is nothing short of amazing.

Les Misérables, which I just got back from at the Marian Theatre, was a treat. If you go to the PCPA site (probably until mid-July at least) you can see a Flash slide show of the various parts of the performance. They don’t allow any recording during a performance so I can’t show anything here but note carefully how the stage is used. For this performace, the stage was largely empty and various sets moved in and out through a back door. Seamless changes made while performers were keeping the action going in the area between the stage and the audience.

For those who may not have seen it, Les Misérables is a musical where not only the songs are sung but the entire dialog is sung. It is set in France around the time of the Revolution and follows the intertwined lives of several people over a number of years. The musical closely follows the work of the same name by Victor Hugo. WikipediaWeb Link does a better job of summarizing the work so I won’t repeat that here.

The show will be running through 10 May 2009 at the Marian Theatre and then from 11 June through 12 July 2009 at the Solvang Festival Theater. If you plan to be in the California Central Coast or need an excuse for a trip here. If Le Miz doesn’t suit you, then 17 Jul-1 Aug 2009 at the Marian Theatre and 7-23 Aug 2009 at the Solvang Festival Theater PCPA will be putting on The Music Man. I have my tickets. 🙂

What does this have to do with computers? Absolutely nothing! But, it’s my blog and I enjoyed the show so much I just had to give others the opportunity to know it exists and maybe equally enjoy it.

One note: The first act is two (2) hours long. Don’t drink several glasses of water before going!

Any Servicemember Mail Suspended

As ex-military, I support supporting the troops. One of the ways of doing so, widely written about on the Web, is to address a letter to “any servicemember” or “any wounded-recovering warrior”. While they were nice programs when they were originated, it turns out that after 2001 these programs were suspended. This suspension has not been widely reported however. In a similar vein, the Dear Abby program has also been suspended. There appear to be no current plans to reinstate these programs any time soon.

Any mail addressed as above should be rejected by the Post Office but, if it happens to make its way to military channels, the mail will be rejected there.

[Information source: Retiree Times (a Lee Central Coast Newspaper), Vol 9, No 2, Spring 2009]

Where Did AddThis Go?

For most of June 2009 CKnow had an AddThis button in the footer of the site. This button allowed users to highlight pages on various bookmarking sites. We did not get many uses but it was an interesting diversion.

Today (10 July 2009), however, I removed all instances of the button from the site and cancelled the account. Why? They started inserting a Flash tracking object to the site without prior warning or permission. Now, to be fair, this object might have been mentioned either directly or by implication in the various agreements they have but I usually read those and don’t recall reading about it.

Even so, since the button did not include this Flash tracking when I first put it on, the change was an unpleasant surprise. I noticed it when viewing the site and NoScript (a Firefox add-in I can’t live without) blocked the call to the clearspring.com website.

If you use AddThis, you might want to watch your website and consider if this sort of action agrees with you and then either leave their button or not as you decide.

Ultimate Windows Tweaker

Every now and again while wandering around the Internet I find something very interesting. Found something today [7/17/2009] that is outstanding for tweaking Windows Vista and Windows 7.

Called Ultimate Windows Tweaker and now in version 2, this is an excellent free program that allows you to make many changes to how both Vista and Windows 7 works. By putting in the right tweaks you can make your system both faster and perhaps more stable. No need to do any registry editing or searching for the proper little dialog box hidden deep into some menu structure; just click on some boxes, click apply, and the changes are made.

For safety you can even create a checkpoint before applying the changes and if you really get lost, there is a restore defaults button that takes you back to Windows as it was installed for any of the changes the program can make.

Even better, no installation is necessary. Just download the ZIP file, open the archive, and run the program.

If you want to give it a try, The Windows ClubWeb Link website has the latest version.