Can I Do This? Grab Guild Info from Guild Armory Webpage Using XML?

[RavingNoah]

Uploaded with ImageShack.us

[Please Skip Down If You Just Want To See The Question]

Above you will find just a quick reference photo of my main character guild on hapless Blackrock offshoot, Garithos (since 2006!)...infamous for its low population, typical lop-sided Horde to Alliance ratio, and consequent lack of overall raid and pvp success. But, sadly...it's my home-within-my-home.

[Warning: This Is Preamble]

Recently, I have started to see (once again) a drop-off in guild attendance...even by my closest friends (i.e., jobs, women, Rift, a nostalgic resurgence of D&D table-gaming, Judge Judy re-runs). This makes running a guild a depressing affair. Sure, Blizzard incorporated the Guild Tool making the act of recruiting, if not the end-result, somewhat easy. But, as I thought about what I needed to do to prepare for having :: shudders :: PuGs amongst my people...I began to realize that if I didn't put into place a workable ranking method, I'd be pulling my hair out once the roster started filling up. Why? Because there are always demands that come from running anything involving those hairless monkeys we call people.

In any case, I realize that I have some goals. Mostly, I want to ensure that members who are contributing guild experience (GXP) are valued in the degree in which they infuse the group with it. But, there is a degree of "what have you done for me, lately" buggering up the issue...because I have some friends who were very excited in December and January who have a high value of GXP attached to their toon who nevertheless I haven't seen in four months. So, I'm devising a method by which GXP is the value at the heart of the ranking method, but how long it's been since logging in, as well as whether or not there is any weekly GXP under their toon's belt will factor together to hopefully create a system of rewards and penalties that I can rely on as I open the guild up to strangers. In this way, at least...some kid who plays this month isn't disadvantaged by a historical number one friend of mine has from six months ago...this is the sort of thing I mean.

[ So...here's the question. ] My guild site in the new WoW Armory lists some key information in a helpful list right on the webpage...and this would be the greatest starting place for me to shorten the time it would take to maintain such a ranking method. I looked at the list, and it is presented in XML... Can I, using the web address as the source file, create an XML batch file that queries (or scans, or lists, or whatever the most appropriate word for this is) the address, locates the information I need, and then sends it to a location (either my desktop, or to another XML sheet to do the ranking calculation before spitting out the functional result into a form that I can then upload to my guild site...like, on the fly).

The address is literally : Roster - Game - World of Warcraft

And I can't help feeling like this would be so easy to do...

[RavingNoah]

Ah, c'mon...I just want to know if it's possible. Even a noob crack would be preferable to board silence...lol.

I think I know how Kitty Genovese must've felt...

[Tim]

Yup, reading websites, grabbing XML, and then doing whatever you want with it... is something done over a bazillion times. Definitely possible. Hard? Nope. Requires programming skills like PHP? Yup.

[RavingNoah]

Grazi, :: bows ::

At least I'm not crazy. Now...where's that PHP book I saw lying around here...and maybe some 'eye of newt' or 'bat wings.' I have no idea what I'm in for...

[Nghtmr9999]

Ahh yes good old PHP. You have fun with that

[RavingNoah]

Originally Posted by Nghtmr9999

Ahh yes good old PHP. You have fun with that

I don't suppose there's an online resource that is particularly suited to the lay-coder that comes to mind?

[Nghtmr9999]

I'm going to take a page out of the good ol' Scrum Agile Development Book
(Scrum (development) - Wikipedia, the free encyclopedia)


You have your end goal in mind, getting information from a website and placing it into a XML sheet. So start breaking it down into little pieces.

First, you know you'll need some way of connecting to and reading a webpage. So Google some code to do that. Afterwards, find some code that will plug that stuff into a XML file. Finally, Google some code that will do your calculations for you.


Based on what you want to do, you aren't just limited to PHP. It sounds like you could use another language like C# to do what you want.


I know my way around C#, and I've been using PHP right now for my job*. If you need assistance, feel free to ask =D

Also, subversion control is one of your best friends. I'm going to start using it more frequently for my personal programming projects.


*sidenote: one of my co-workers turned me to the "dark side" of programming and got me hooked on emacs. I'm still waiting for the cookies.

[Zub]

Definitely possible. cUrl is your friend.

Here's some old code from one of my signature tools, might be of help.
Obviously you need to find the right url to call to get the data for your guild.
Also, IIRC most pages return an XML layout of the data, so it's easily extracted, rather than scraping a full html page.

PHP Code:

      // grab from armory
    $fp = fopen('curl_wowparser.txt', 'w'); //trace file

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);  //url of your guild armory page
    curl_setopt($ch, CURLOPT_REFERER, $referrer); //base armory url
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2");
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);        
    curl_setopt($ch, CURLOPT_VERBOSE, 1);
    curl_setopt($ch, CURLOPT_STDERR, $fp);
    $this->data = curl_exec($ch);
    $http_code = curl_getinfo($ch, CURLINFO_HTTP_CODE);
    curl_close($ch);
    fclose($fp); //closing trace
        
    //die("ERROR: ".$http_code." - ".$url);
        
    if ( ($http_code == "") || ($http_code == "200") ) return $this->processData(); 
    else return 1;     //curl error; 


the page data will be in the "data" variable.
Hope this helps.
-Zub

[RavingNoah]

Guys, thank you SO much.

I'm going to start playing with your suggestions. I'll let you know how it's turning out.

Yay...stuff to play with... :: skips merrily onward ::

In other news, a tech support guy from Wowstead told me .php/.xml/mysql was not supported, but if I made a mock-up, he'd pitch it to the devs for possible inclusion (if other people would use it).

[RavingNoah]

"Can I play with the jump rope?"

"Yes, you can. But first, grab this pulse oximeter, enter the readings into FARK and then conduct a patch aortoplasty Coarctation repair on Mrs. Crabapple in that room, there..."

Sigh...sniff...I just finished running the supposedly easy-to-use idiot-proof Web Development Installer for Windows with its "new and improved" support for PHP...after trying to install the MySQL, Apache HTTP, and PHP tools from a book I bought a year or so ago which failed at the Apache installation, presumably because it wasn't made for a Windows 7 64 environment...and, naturally, despite the positive publicity statements the devs were gushing during an hour and half long podcast about its ease-of-use, it also failed to install the MySQL tools that, at least according to my book, I'll need to start toying with the dumb little apps I want to try building to handle clan data, Guild XP and DKP toon evaluation...

I have a drive that currently houses my old Vista OS that I would just love to stick Ubuntu or whatnot on it, just so I could use the installers you get from Apache, MySQL, and PHP.net without the hassle that comes with trying to do this on my Windows machine...but fercrissakes...what level of coding diety-ship-osity-ness is required?

Please help assuage my bruising heart...give unto me a link to creating a development environment setup that is appropriate to my meager skill level but will still let me play...just a little bit...with the language of kings...lol.

Okay, it's a little much...but damnitall if my frustration level hasn't hit orange thirteen times today...


...ergh.