In response to Satinder's post about growth of the field, I did a little experiment. Google scholar allows for date searches. (Citeseer has similar information, but I couldn't figure out a way to work with it.) I searched for the phrase "reinforcement learning" and counted the number of hits in each year. Google scholar also creates a "key author" list for each search, so I included that information as well.
Before 1963, Google scholar lists 42 papers, but it seems nearly all of them are errors in detecting the year of publication (using the page number instead). After that, things muddle around in the single digits until the 80s when the UMass folks begin attracting attention to the problem. The learning automata and GA folks are pretty visible during this period.
In the 90s, the per-year paper count swells from 100 to 1000. Non-UMass pockets are clearly visible. Multiagent folks are out in force by the end of the decade. In the 2000s so far, we're seeing a spread to the students (and students of students) of the pioneers as well as paper counts up above 2000.
Plotting the paper counts, it doesn't look so much like an exponential as a quadratic with a weirdly high peak in 2005. In any case, Satinder's intuition that things have grown substantially appears valid. :-)
-----------------
1964 1 P Wasserman - F Silander - M Foundation
1965 6 M Waltz - K Fu
1966 5 K Fu - Z Nikolic
1967 1
1968 4 K Fu
1969 3 J Carlyle
1970 12 K Fu - J Mendel - E Davison - G SARIDIS - R McLaren
1971 5 J Albus
1972 2
1973 4 J Justice - J Shanks - J MENDEL - J ZAPALAC
1974 3 R Monopoli - J Mendel
1975 5 J Holland
1976 1 P Verschure - T Voegtlin - R Douglas
1977 0
1978 1
1979 4 G Saridis - C Van Rijsberg...
1980 2
1981 8 A Barto - R Sutton - P Brouwer - G Saridis - W Croft
1982 2 R Sutton - A Barto - D Reilly - R Williams - L Cooper
1983 4 P Schweitzer - D Lenat - S Hampson - D Kibler
1984 12 P Young - J De Kleer - J Brown - W Croft - R Thompson
1985 8 R Korf - N Cramer - J Gould - R Levinson - A Barto
1986 13 R Sutton - P Kumar - C Anderson - P Varaiya - A Barto
1987 16 D Bertsekas - D Goldberg - J Richardson - D Ackley - R Korf
1988 38 B Widrow - R Sutton - M Hoff - P Sahoo - S Soltani
1989 63 K Narendra - T Kohonen - D Goldberg - M Thathachar - C Anderson
1990 151 R Sutton - P Maes - P Werbos - V Gullapalli - A Benveniste
1991 198 R Sutton - S Whitehead - D Ballard - D Chapman - C Lin
1992 275 C Watkins - L Lin - P Dayan - S Mahadevan - R Williams
1993 348 L Kaelbling - A Moore - C Atkeson - N Lavrac - P Dayan
1994 467 M Littman - M Puterman - S Haykin - G Rummery - J Boyan
1995 530 D Bertsekas - S Russell - A Samuel - G Tesauro - P Norvig
1996 695 L Kaelbling - M Littman - D Bertsekas - A Moore - J Tsitsiklis
1997 803 M Tan - H Kitano - M Dorigo - M Asada - Y Kuniyoshi
1998 990 J Hu - C Claus - M Wellman - R Parr - C Boutilier
1999 1100 R Sutton - T Dietterich - S Singh - D Precup - J Rennie
2000 1190 S Singh - K Doya - R Sutton - M Littman - W Smart
2001 1290 M Littman - S Dzeroski - K Driessens - L Peshkin - P Stone
2002 1490 M Kearns - S Singh - K Doya - W Smart - B Hengst
2003 1700 A Barto - S Mahadevan - R Brafman - D WOLPERT - C Guestrin
2004 1770 A Ng - Y Shoham - R Powers - T Grenager - J Si
2005 2060 A Barto - S Singh - D Ernst - S Collins - M Bowling
2006 1830 S LaValle - P Stone - J Peters - S Whiteson - Y Liu
2007 2000 P Stone - S Mabu - M Taylor - J Peters - K Hirasawa
2008 347 J Drugowitsch - A Barry - H Tizhoosh - E Courses - Y Liu
Subscribe to:
Post Comments (Atom)
2 comments:
Thanks Michael. This is really really useful. I agree that there isn't really an answer to the question "how big RL is at the moment". I was hoping to spur exactly this kind of data collection. Now that you have done this, I might add to it by querying on other more specific terms, like Q-learning, and Sarsa. It might be a good way to see when things start spreading and perhaps even when specific ideas start declining in usage and reference.
I think a useful tool that could be used to show some interesting search patterns and news visibility is Google Trends. For instance searching in parallel machine learning, neural networks and reinforcement learning gives: http://www.google.com/trends?q=machine+learning%2C+neural+networks%2C+reinforcement+learning
Post a Comment