NAME

CPAN::Forum - Web forum application to discuss CPAN modules

SYNOPSIS

Visit <http://www.cpanforum.com/>

DESCRIPTION

This is a Web forum application specifically designed to be used for discussing CPAN modules. At one point it might be adapted to be a general forum software but for now it is released in the hope that people will help improving it and by that improving the <http://www.cpanforum.com/> site.

Features

• Posting only by authenticated users.
• For registration a valid e-mail is required.
• Username and password should be in lowercase and unique.
• Every poster will have to give
  1. name of the group
  2. Subject
  3. Content
  4. (Future) A unique id to the post and the post made in response to.

     This reference id will be NULL for new posts. At the beginning responses will have to maintain group but can change the subject and will have to write new text.

• Later we might enable changing the group to a related group.

  (e.g. a message about a module can get a response in a group to which this module belongs to)

• Later we might enable total random change in the groups.
• We make sure the links are search-engine friendly
  • /posts/ID (link to a post) - /threads/ID (link to a thread)
• We provide RSS feed of the recent posts belonging to any of the groups.
• We'll provide search capability with restrictions to groups.

Authentication

Shared authentication with auth.perl.org? I once tried to do this but then for some reason I could not finish the process. Maybe later we'll want to enable our users to use their auth.perl.org identity. Maybe we can do it also with PerlMonks. Right now we have our own registration and login mechanism.

INSTALLATION
Apache
This is the configuration of my Apache server on my notebook

AddHandler cgi-script .pl

        <VirtualHost 127.0.0.1>
            ServerName cpan.local
            DocumentRoot                /home/gabor/work/gabor/public/dev/CPAN/www/
            ScriptAliasMatch ^/(./.)  /home/gabor/work/gabor/public/dev/CPAN/www/cgi/index.pl/$1
            DirectoryIndex              cgi/index.pl
        </VirtualHost>

        <Directory "/home/gabor/work/dev/CPAN">
            Options Indexes FollowSymLinks ExecCGI
            AllowOverride None
            Order allow,deny
            Allow from 127.0.0.1
        </Directory>

hosts
For local installations in /etc/hosts I added:

127.0.0.1 cpan.local

That way, I have a totally separate virtual host just for this application.

In a real setting probably you'll have something like www.cpanforum.com pointed to your server.

Install the perl code

        perl Build.PL
        ./Build
        ./Build test
        ./Build install dir=/path/to/install
        cd /path/to/install

        chmod a+x www/cgi/index.pl  (needed only if you work out of the repository)
        chmod a+x db/forum.db       (or whatever you need to make sure the database is writable by the web server.

Finally, manually edit the www/cgi/index.pl file and set the sha-bang to the correct one

Setup the database
In the directory where you installed the modules create a file called CONFIG (see t/CONFIG for an example). Having the following fields:

        username=        The user name of the administrator.
        email=           The E-mail of the administrator.
        password=        The password of the administrator.
        from=            The Email address to be used as the from address in the 
                         messages sent by the system.

You will be able to change all these values later from the web interface but we need to have the first values.

Run

perl bin/setup.pl

(you can now delete the CONFIG file)

Run

perl bin/populate.pl

(this will fetch a file from www.cpan.org and might take a few minutes to run)

CPAN_FORUM_URL
For some of the tests you'll have to set the CPAN_FORUM_URL environment variable to the URL where you installed the forum.

Changes
v0.11 Search for users Unite the serch methods Accept both upper-case and lower-case HTML tags and turn them all to lower case tags when displaying Accept <a href=> tags for http and mailto Admin page Admin can change "From" e-address Enable <i>, <b> <br> and <a ..> with <p></p> pairs Remove the selection box from the post interface as it was not used there. Put the search form on the home page as well. Admin can change e-mail address of any user Add paging

v0.10_02 <p>, <br> enabled Add link to Kobes Search Improve full text search for posts Add capability to search for module names

v0.10 - markup improved, bugs fixed

v0.09_05 - POD cleanup (Shlomi Fish)

• More tests
• Start using Parse::RecDescent

v0.09_04

• Before writing a new post instead of showing a list of all the modules now the user first will search for a module name. post link should give a search box that will let the user search within the names of the modules. The result should be a restricted list with only a few module names in a pull-down menu like we have now. The search is a regular SQL LIKE search and we add % signs at both ends of the typed in word.

TODO

• Enable people to subscribe to all messages or all thread starters or all followups (probably group = 0 will do it) - Add announcement service (check for new versions of modules and send e-mail to those whom are interested in announcements. - Put time ellapsed since post instead of date of post, (3 min. 6 hours 3 days ago) make this configurabel: User can say after N days passed the date should show, before that the ellapsed time - Admin: hide a posting - Admin: delete a user - Admin: add a new module manually - Script that populates databse should not lock the whole database for a long time Maybe it should fetch all the data to memory and work there. - make paging available responses 1..10, 11.20, etc,
• Decide on Basic Markup language and how to extend for shortcuts opening tag for code: <code[^>]*> but right now only <code> should be accepted closing tag for code: </code>
• check all submitted fields (restrict posting size to 10.000 Kbyte ? - Improve text and explanations.

clean up documentation

add indexes to the tables ?

show the release dates of the various versions of a module so it is easy to compare that to the post.

Authentication and user management process: - new user comes to our site we give him a cookie, when he wants to login we offer him -- login using the auth.perl.org credentials -- login using XYZ credentials -- create local credential

-- For auth.perl.org --- redirect the user to auth.perl.org wait till he logs in there (maybe even creates the new account) --- sets the preferences --- comes back --- we can fetch some of the information from that user --- we need to keep the user_id received from auth.perl.org for later identification of the user --- while we tell the user we would like to get the username/fullname/e-mail address from auth he might not want to give, for this case we should have our way to update the locally updated username, full name and validated e-mail address.

-- For local credentials we need the user to give us username/password/fullname and validated e-mail address.

We have to make sure that usernames which are displayed don't collide. Maybe we should use separate fields for usernames from various sources and when displayed we might prefix it auth:gabor, local:gabor etc. Not nice, any better way ?

Subject field: - <= 50 chars - Can contain any characters, we'll escape them when showing on the web site

Text field: - No restriction on line length, let the HTML handle that part - The text is divided into areas of free text and marked sections

In order to avoid accepting postings today that will break when we add more tags, we will reject any submission that is not correctly marked up.

• Pages: new mesage: EDITOR; PREVIEW + EDITOR show: POST response: POST + EDITOR; POST + PREVIEW + EDITOR

        thread:          P1 + P2 + .. Pn
        thread response: P1 + P2 + .. Pn + EDITOR;   \
        P1 + P2 + .. Pn + PREVIEW + EDITOR;

When the EDITOR comes up first the subject should be filled by the subject it is answering to or empty for new message.

The PREVIEW and the EDITOR should be filled by the same information, though within the editor we don't need the parent id and similar to be shown.

• Enable some administrator to mark a message (or a whole thread) to be hidden (database already has field)
• Enable some administator to mark a group to be - hidden (messages don't show up) - frozen (cannot add new message but still can see the earlier messages) Critical part: make place for this in database (status field)
• Administrator (or even the author ?) should be able to move a message from one module to another module or group.
• Enable administrator to ban a user (mark in the users database to disable the user) Hmm, do I really need this ? maybe as I cannot just delete a user. (added a status field that is not used currently)
• Replace the /post/number link by /post/TITLE_OF_POST ???

OK, so we have listing in places like

            /
            /dist/Distro-Name
            /users/USERNAME
        
            /all  Can be a name for all the posts so we don't need to put any other 
              information immediately after the first slash  maybe it should be 
              /home ?

        /dist/Distro-Name/start/count
            /all/start/count

We'll also have some search facility that will be a post operation and

            /posts/id          show a post      (show post         )
            /new_post/         start new post   (            editor with module list)
            /new_post/Module   start new post   (            editor  no  module list)
            /response_form/id  start a respones (show post + editor)

From the forms we have post methods so no need for URL munging process_post => (show previous post)? show editor + show preview

• make the page size (for paging) user configurable

        A user can ask to be notified upon the following events per
        distribution. subscriptions: uid, gid, (all), (starter),
        (participate) 1) All messages

        All the messages execute QUERY: select uid FROM subscription WHERE
        gid == disto and all.

        2) Thread starters Thread starteres (where id=thread) execute QUERY:
        select uid FROM subscription WHERE gid == distro and starter

        3) Followup messages in a thread he participated already Every
        message (well, except thread starters) execute: QUERY: select uid
        FROM subscription - there is a post with the same thread id as of
        this post which was posted by a user which has a subsciption
        (participate)

        4) People who are not subscribed to All messages (1) when seeing an
        interesting posting they can say: send me all followups. uid,
        threadid

        She can set up such notification on a per module basis or for all
        the modules.

        After logging in the user can modify his "subscriptions" to such
        notifications. The notification will be sent out from an e-mail
        address such as noreply@bla.com which will discard any message sent
        to it. The message will contain the text of the post, a link to the
        post_response page, a link to view the whole thread and an e-mail
        address in case someone wants to complain/whatever.

• Subscription (notify) management:
• /mypan will be the url to get and set all the configuration information. It will list all your current subscriptions and you can enable/disable them. Normally this will show only distributions you have some kind of a subscription.

In addition from /mypan there will be a way to ask to add a new subscription by selecting the name of a module and the initial subscription parameters to it.

In addition when displaying the list of all the messages to a specific module, logged in users will see their current subscription to this module (even if that is empty).

• Fix Installation
• when installing one might need to be root, in order to set the permissions correctly ?
• as user www fetch the module list file, unzip it in the db directory (as this is the only directory we can write to) and run the populator
• on a new installation, change the ownership of directories (or at leas tell the user to do so)
• Write comprehensive test suit
• Reply within a thread

When replying to a post within a thread we might want to open the editor window in the middle of the thread, just below the post I am responding to.

• Make the Session use the database instead of plain file
• Make autoposter of new version announcements work

A script that will send an announcement on the new version of every module to the list I think this is done as a script listening in the cpan-testers mailing list, though it might be one similar to bin/populate.pl

• make sure links that are relevant for distros don't show up on pages which don't belong to distros. (e.g. a link to search.cpan.org/dist/CGI is ok but a link to search.cpan.org/dist/General is not)
• Sometime we'll want to post a message in more than one group, e.g. now I'd like to know how to use CGI::Session with DBD::SQLite. I might want to post the message on more than one list at the same time as this is related to more than one module. Porbably if I need to chose one I'll select CGI::Session as I am trying to use that but it might be a nice feature. Maybe I need to tell one module as the main group and then have a way to associate a few more modules with the posting.

This can be done by de-coupleing the name of the distribution from the posts table for all the distributions or we can add such an extra table for the additional distributions so there will be a leading distro of the thread.

• Getting the listing of all ~7000 module names takes a long time. I should profile it. 1) write a small script that will run the relevant code on the command line, 2) time this 3) look at the size of the output 386K -> it won't fly, you can't have such a page on the web. Other solutions: - type in the name - search for the name - currently we'll keep a separate file called db/modules.txt with a listing of all the distros. This make page generation in 1-2 sec instead of 7-8. Obviously there is a problem we'll have to fix.
• Create a group for - each Distribution (DONE) - Some bigger groups (eg. databases, testing, ) maybe put each distribution under one or more of the groups too - General and other special purpose groups such as News (for the site) where only "administrators" can post.

I am not sure if I have to keep all these things in one table and if the same form has to serve for creating messages in both distros and categories.

• Database or plain files ?

I think every information should be in the database but then we might want to generate static pages from the posts and discussions in order to reduce the need to fetch information from the database. Hmm, it sound faster but we'll probabl want to build the pages on the fly anyway so maybe it does not improve anything. We can start off by totally dynamic pages and then see if making them static will reduce the load on the server. First we'll have to have load on the server. :-)

• Check if the technique we use to remember the last request before login cannot cause some security problem such as remembering the last request of someone else who used the same machine recently ?
• xml - provided
• favicon.ico and a banner image would be good

Shlomi: The Forum uses cgiapp_prerun to set the mode according to the PATH_INFO instead of using a mode_param code-reference. This causes a lot of warnings in the logs, and doesn't really belong in cgiapp_prerun.

It cannot be hosted on a URL except for its own virtual host, as it uses absolute URLs. ("/login/", "/register/", etc.) A better idea would be to track the path that the web-server gives (it's in one of the environment variables) and then to construct a /cpan-forum/login/ /cpan-forum/register/ etc. path. (or use relative URLs).

TEMPLATES
root templates:

about.tmpl change_password.tmpl faq.tmpl groups.tmpl help.tmpl home.tmpl internal_error.tmpl login.tmpl module_search_form.tmpl module_select_form.tmpl mypan.tmpl notes.tmpl posts.tmpl pwreminder.tmpl search.tmpl users.tmpl register.tmpl threads.tmpl

every root template should INCLUDE -> head.tmpl navigation.tmpl footer.tmpl

groups.tmpl -> links.tmpl listing.tmpl (list of messages within one group) home.tmpl -> listing.tmpl (list of messages in all the site) posts.tmpl -> links.tmpl message.tmpl message.tmpl editor.tmpl (single message with or without the editor with or without a preview pane) search.tmpl -> listing.tmpl (list of messages resulted from search) threads.tmpl -> links.tmpl users.tmpl -> listing.tmpl (list of messages of one user)

Non root templates: links.tmpl - a bunch of links to search.cpan.org and similar places (specific to one distro) listing.tmpl - can show the titles of many messages message.tmpl - can show one message (or a preview message) naviagtion.tmpl - head.tmpl - footer.tmpl -

Use this for mapping: grep INCLUDE *| grep -v navigation.tmpl | grep -v footer.tmpl | grep -v head.tmpl

METHODS
cgiapp_init
Standard CGI::Application method.

Setup the Session object and the default HTTP headers

setup
Regular CGI::Appication method to setup the list of all run modes and the default run mode

cgiapp_prerun
Regular CGI::Application method

We use it to change the run mode according to the requested URL (PATH_INFO). Maybe we should move his code to the mode_param method ?

autoload
Just to avoid real crashes when user types in bad URLs that happen to include rm=something

home
This the default run mode, it shows the home page that includes the list of most recent posts.

redirect_home
Just to easily redirect to the home page

about
About box with some statistics.

internal_error
Gives a custom Internal error page.

Maybe this one should also receive the error message and print it to the log file.

load_tmpl
Semi standard CGI::Application method to replace the way we load the templates.

login_process

• Processing the information provided by the user, - calling for authentication - setting the session
• redirecting to the page where the user wanted to go before he was diverted to the login page

logout
Set the session to be logged out and remove personal information from the Session object.

groupselector
It is supposed to show the form to write a new message but will probably be a redirection.

response_form
Probably obsolete.

posts
Show a post, the editor and a preview - whichever is needed.

process_post
Process a posting, that is take the values from the CGI object, check if they are acceptable and try to add them to the database. If anything bad happens, give an error message preferably by filling out the form again.

threads
Show all the posts of a thread.

dist
List last few posts belonging to this group, provides a link to post a new message within this group

users
List the posts of a particular user.

mypan
Planned to be the manager for the notify subscription, currently not in use.

rss
Provide RSS feed /rss latest 20 entries /rss/dist/Distro-Name latest 20 entries of that distro name

notify
Send out e-mails upon receiving a submission.

_text2mail
replace the markup used in the posting by things we can use in e-mail messages.

ACKNOWLEDGEMENTS

Thanks to Offer Kaye for his initial help with HTML and CSS. Thanks to Shlomi Fish for some patches. Thanks to all the people who develop and maintain the underlying technologies. See <http://www.cpanforum.com/about/> for a list of tools we used. In addition to Perl of course.

DEVELOPMENT

Subversion repository is at <http://svn.pti.co.il/svn/cpan-forum/trunk/>

There is a mailing list to see the commits to the repository: <http://perl.org.il/mailman/listinfo/cpan-forum-commit>

Discussion of this module will take place on <http://www.cpanforum.com/dist/CPAN-Forum> If you need help or if you'd like to offer your help. That's the right place to do it.

BUGS

Please report them at <http://rt.cpan.org/>

LICENSE

Copyright 2004-2005, Gabor Szabo (gabor@pti.co.il)

This software is free. It is licensed under the same terms as Perl itself.