A couple of years ago, librarians at the University of Huddersfield made a connection. They realised that analysing the electronic trail left every time a student swiped into the library, borrowed a book or looked something up online, and putting it together with other student records could not only help to improve library services but also answer more fundamental questions about the way students learn. Was use of the library, for example, related to how well students performed academically?

The answer proved emphatic. By plotting library usage against academic achievement they discovered that students who did not use the library were more than seven times more likely to drop out of their degree than those who did.

From the next academic year, collecting and analysing data will become a formal part of Huddersfield's teaching and learning strategy. By putting together information on what students do, such as whether they attend lectures and how they perform, including their particular strengths and weaknesses, the university wants to make staff and students more aware of what works – and to respond accordingly. It will affect the way staff design the curriculum, plan classes during the term and make decisions about the kind of learning support needed by different groups of students or individuals.

"We are looking at it in terms of student achievement," says Cath Ellis, director of teaching and learning in the school of music, humanities and media at Huddersfield. "How can we get more students out of the 2.1 range into a first class?"

Huddersfield is not the only university to realise data's potential. While next term's students will be better informed than any previous cohort about the institutions they have chosen to attend, thanks both to social media and new key information sets allowing them to compare different aspects of university life, institutions will also be better informed than ever about their students.

While universities have routinely collected information about students for years – from their family backgrounds to what books they take out of the library – increased computer power and better digital skills now offer the possibility to piece it all together. It could fundamentally change the way institutions operate – as well as raising challenging ethical and privacy issues.

"It's almost waste stuff, generated as a by-product of communications, and previously we did nothing with it," says Rob Englebright, programme manager at Jisc, which champions use of digital technologies in education. "Now we can look at it and form patterns."

What patterns universities choose to look at, and how they use what they find, varies. Some are using such so-called data analytics for more effective marketing, others to identify the most efficient way of putting together a research grant, others to manage staff performance, and many to help prevent students dropping out.

One of the first institutions to use data to help retention was Purdue University in Indiana, which four years ago introduced a traffic light set of signals every time students logged into the course website. This warned them if they were likely to fail (a red signal), when compared with the behaviour of previous students. A red signal came with suggestions of how to get back on track to green, through attending help sessions or reading more.

Since then, a few UK universities have developed similar systems, including the University of Derby. It is looking at the way students engage with the university generally, including not only how they interact with the virtual learning environment, but also whether they are captain of the rugby team and their use of car parking.

Jean Mutton, student experience project manager at Derby, says: "Staff are telling us that this sort of information collected together would be invaluable. It would help them understand where students are coming from and the challenges they are facing." She says it will also help universities later to engage with alumni, as well as being useful for academics writing references.

So useful is analysing student behaviour expected to be in future that researchers at Loughborough University are now hoping to market the internal system they have developed – Co-Tutor – nationally and, eventually, internationally. Melanie King, head of the centre for engineering and design education at Loughborough, says they intend to launch a commercially viable system next year.

Co-Tutor focuses on tracking individual students' relationships with their personal tutors and placement supervisors, identifying how often students – and staff – are turning up to face-to-face meetings.

The university is also researching the possibility of incorporating analysis of emails and other text interactions between staff and students to detect how negatively a student feels about the course, and therefore how much they are at risk of quitting. King says: "We were trying to encourage staff to be more organised, but it didn't take long for them to realise the benefits."

Ellis acknowledges that the idea also presents challenges for staff because analysing data monitors their behaviour as well as that of their students. It also potentially increases their workloads. She cites anecdotal evidence that some institutions are bombarding staff with data about their students without helping them to work out what it means, or what they should do with it, which then raises questions about where liability lies if no one acts on the information.

And then while many students are spurred on by seeing that their behaviour is not up to scratch, others are not. "Just because you have this data and can draw conclusions from it doesn't mean you should show it to students or tutors, because it could do more harm than good," Ellis says. "We are still evaluating the impact it has."

Englebright acknowledges that looking at big data sets involves "scary ethics" because it influences how universities allocate resources. One danger is that they could identify the demographic groups most likely to drop out and stop recruiting them, or decide not to waste money on supporting students likely to leave.

"If a student hasn't attended the library, do you intervene to prevent them failing, or in a cynical way do you say we will get rid of them early?" he asks. He compares it to Minority Report, the film in which Tom Cruise plays a futuristic cop able to use data patterns to predict crimes and stop them before they happen.

While most universities have strong ethics committees and good track records in protecting data because of its use in research, he is concerned about other learning providers coming on the scene, and the effects of growing competition and tighter financial margins.

"There are some fantastic opportunities to do stuff with the data and bring people on, but there is the potential for nasty people to do stuff as well," he warns.

The University and College Union is wary about using quantitative data sources as a performance management tool. "By their very nature, such sources of data do not take into account a range of other contextual factors which are of critical importance when making judgments about individual staff members' work," says its president, Simon Renton.

But while the National Union of Students agrees that context is important and wants institutions to share data with students' unions and to prevent it getting into the hands of third parties, it is generally behind the idea of data collection. "Institutions taking the initiative to look at what students' experiences are and using that to improve them is a good thing," says Colum McGuire, vice-president, welfare.

The influence of large data sets will rise, especially with the growth of massive open online courses (Moocs), which let institutions teach thousands of students over the internet, collecting data on them as they go. Researchers are also investigating how to bring in data about students drawn from social media, such as Twitter and Facebook. And this year Oxbridge academics will publish the results of a controversial research project plotting tax records against data from the Student Loans Company, which will show the relationship between institutions, subjects and graduate earnings.

Englebright stresses that collecting information is not what matters, but how it is used and who uses it: "It's all very well having the data, but unless you can do something with it, it's just noise."

• This article was amended on 5 August. Loughborough University is looking at the possibility of analysing emails, but is not already doing it.